Mathematics Achievement Academies
2011 Evaluation Report
to
Chevron U.S.A., Inc.
January, 2013
Mathematics Achievement Academies – 2011 Evaluation Report
Prepared for
Dr. Julia Olkin, Principal Investigator
Co-Director, Center for Math Education and Research
by
Dr. Beth Yeager, Lead Researcher
Institute for STEM Education
CA State University, East Bay
Dr. Eric A. Suess
Department of Statistics and Biostatistics
CA State University, East Bay
With contributions from student assistants
Hao Le
Tony Tran
Aida Yazdanparast
Peter Cummings
Qiongsi Dong
2
Abstract
In Summer, 2011, the Mathematics Achievement Academies (MAA) program
began Year 2 of a three year program, primarily funded through the generosity of
Chevron, U.S.A., Inc., to support middle and high school students in enhancing their
capacity to successfully complete Algebra I, Geometry and Algebra II (Summer, 2012).
MAA offered a total of 39 academies in 2011, 22 in algebra and 17 in geometry, with a
total of 470 students completing the academy. The program had a 68% retention rate in
Algebra and a 72% retention rate in Geometry.
Findings from the evaluation of the 2011 MAAs, support the conclusion that the
program is having a positive impact on average student growth in content knowledge in
both algebra and geometry. Based on analyses of assessments for all students
enrolled in the program, raw percentages show a growth in algebra content knowledge
from a mean of 40% for all students taking a pre-test to 52% for those students taking a
post test. Findings also indicate growth in geometry academies from a mean of 37% for
all students taking a pre-test to 50% for those students taking a post test. When
findings are disaggregated by district and class, they show that 16 out of 22 classes in
algebra showed statistically significant improvement, and in geometry, 15 out of 17
classes showed statistically significant improvement.
Even if students attend only one year of summer academy, algebra results
support the summer academy approach that emphasizes preparation rather than
remediation. Additionally significant are the differences in growth in geometry
academies. The findings from this analysis of the introduction of the geometry
academies in 2011 supports the positive potential and consequences for student
learning of a long-term approach to student preparation for entering gatekeeper math
courses.
3
Table of Contents
Executive Summary
5
Introduction
7
Program Design Overview
8
Project Impact
9
Project Participants
Findings – Pre/Post Math Content Tests
9
13
Algebra - Pre/Post Test Findings by District and Class
17
Geometry – Pre/Post Test Findings by District and Class
22
Findings – Pre and Post Student Self-Assessment of Item Difficulty
– Algebra and Geometry
Concluding Remarks
Appendices
27
32
33
Appendix I - Algebra - Student Content Knowledge Test Analyses
by Class
34
Appendix II - Geometry - Student Content Knowledge Test
Analyses by Class
57
4
Executive Summary
In Summer, 2011, the Mathematics Achievement Academies (MAA) program
began Year 2 of a three year program to support middle and high school students in
enhancing their capacity to successfully complete Algebra I, Geometry and Algebra II
(Summer, 2012) courses. Primarily funded through the generosity of Chevron, U.S.A.,
Inc., through its California Partnership, MAA is administered by the Center for
Mathematics Education and Research (CMER) at California State University, East Bay,
directed by Dr. Julia Olkin and Dr. Phil Duren. In 2011 CSUEB collaborated with
Alameda County Office of Education (ACOE).
Designed to assist in preparing students, who have previously struggled with
math classes, for college-preparatory mathematics, MAA is a four-week (sixty hours)
math-intensive summer program. The interactive classes and activities emphasize
rigorous content, collaboration, creative thinking, and logical reasoning. Instructors for
the academies were primarily drawn from a pool of teachers who had previously
participated in professional development courses offered through Project ACCLAIM (the
Alameda County Collaborative for Learning and Instruction in Mathematics).
In 2011, MAA offered 22 academies (sections) in Algebra to students who would
be entering 9th grade and enrolled in Algebra I in Fall, 2011. The 22 academies for this
second cohort of students reflect an increase of 2 over the previous year’s (2010)
offerings in algebra. In addition, 17 academies (sections) were added and offered in
Geometry for students entering that discipline as 10th graders in fall, 2011, making a
total of 39 academy sections in both algebra and geometry offered to a total of 470
students who completed the academy. The 39 academies were hosted across 13
different middle and/or high school sites for Algebra and 10 different sites for Geometry
in the participating districts.
Participating students were drawn from 11 districts in both Alameda and Contra
Costa counties (with one district in San Benito County). A high number of these
districts include a significant percentage of students who receive free and/or reduced
lunch under Federal guidelines. Free and/or reduced lunch percentage is an indicator
used by schools to identify socioeconomic status. At the same time, many of these
districts have significant numbers of often underrepresented students in STEM
pathways (e.g., Latino, African American). Findings from analysis of student enrollment
show that the 2011 program had a 68% retention rate in Algebra and a 72% retention
rate in Geometry.
Students completed a pre and/or post assessment of math content knowledge at
the beginning and end of the four-week summer session. The assessments in both
algebra and geometry (one for each discipline) consisted of 35 multiple choice items.
Findings from analyses of pre/post student assessments showed overall growth in math
content knowledge in both algebra and geometry academies. Based on analyses of
assessments for all students enrolled in the program, taking a pre and/or post-test, raw
percentages show a growth in algebra content knowledge from a mean of 40% for all
students taking a pre-test to 52% for those students taking a post test. Findings also
indicate growth in geometry academies from a mean of 37% for all students taking a
pre-test to 50% for those students taking a post test.
5
It is also possible to isolate those students (N=295) taking both the pre and the
post-test in algebra. This analysis is able to show growth, for these students, of 3.83
points (between averages). There was a 6.12 difference between the pre-test average
and the post-test averages for students taking both a pre and a post test in geometry,
an increase twice as great as that for algebra.
Finally, while all classes, examined broadly, show growth in mean student
content knowledge, when findings are disaggregated by district and class, they show
that 16 out of 22 classes (academies) in algebra showed statistically significant
improvement. In addition, when findings are disaggregated by district and class in
geometry, 15 out of 17 classes showed statistically significant improvement.
As students took pre and post-tests of content knowledge, they were also asked
to assess the degree of difficulty (simple, challenging, difficult) of each of the 35 items
as they came to it (self-assessment, opinion). Findings from analyses in algebra and
geometry of student assessment of difficulty showed the overall average reported
difficulty for each item decreased between the pre and the post-test. Students
increasingly indicated items as simple by the post-test and decreasingly indicated items
as challenging and/or difficult. In addition, when further analyzed, the findings make
visible a progressive shift between pre and post-tests in both Algebra and Geometry
Academies in students self-reporting from high levels of difficulty of test items to
increasingly perceiving items as ‘simple’, or less ‘challenging’ or ‘difficult’.
Findings from the evaluation of the 2011 Math Achievement Academies, support
the conclusion that the program is having a positive impact on average student growth
in content knowledge, as evidenced by the overall growth in algebra. Even if students
attend only one year of summer academy, algebra results support the summer
academy approach that emphasizes preparation rather than remediation. Additionally
significant are the differences in growth in geometry academies. It is recommended that
additional attention be paid to this finding in Year 3, in order to assess student
participation, opportunities for learning, and the impact of the program on those
students who attended academies for multiple years. The findings from this analysis of
the introduction of the geometry academies in 2011 supports the positive potential and
consequences for student learning of a long-term approach to student preparation for
entering gatekeeper math courses.
6
Mathematics Achievement Academies – 2011 Evaluation Report
Introduction
In Summer, 2011, the Mathematics Achievement Academies (MAA) program began
Year 2 of a three year program to support middle and high school students in enhancing
their capacity to successfully complete Algebra I, Geometry and Algebra II (Summer,
2012) courses. These classes are often considered ‘gatekeeper’ courses. Having the
capacity to complete them, without requiring remedial coursework, can help to ensure
that these students, most from underrepresented groups, will be prepared to enter
college pathways (including California State University, East Bay) as freshmen.
Primarily funded through the generosity of Chevron, U.S.A., Inc., through its California
Partnership, MAA is administered by the Center for Mathematics Education and
Research (CMER) at California State University, East Bay, directed by Dr. Julia Olkin
and Dr. Phil Duren. In 2011 CSUEB collaborated with Alameda County Office of
Education (ACOE), and since 2012, that collaboration is with West Contra Costa USD,
with Phil Gonsalves. The project serves students throughout Alameda and Contra
Costa Counties. Additional financial support from the Dean and Margaret Lesher
Foundation supported further expansion into Contra Costa County, starting in 2012.
Designed to assist in preparing students, who have previously struggled with math
classes, for college-preparatory mathematics, MAA is a four-week (sixty hours) mathintensive summer program. The interactive classes and activities emphasize rigorous
content, collaboration, creative thinking, and logical reasoning. The program is not
meant as a substitute for courses offered during the school year, nor as remedial
support. Rather the summer academies can be seen as a way of enhancing student
capacity in order to successfully access and make meaning from complex content as
they enter these ‘gatekeeper’ courses in high school.
According to documents produced by or for the program, Year 2 was intended to both
build on and expand what occurred in Year 1 (O’Driscoll, 2010; Olkin, Nov., 2011). To
that end, MAA offered 22 academies (sections) in Algebra to students who would be
entering 9th grade and enrolled in Algebra I in Fall, 2011. The 22 academies for this
second cohort of students reflect an increase of 2 over the previous year’s (2010)
offerings in algebra. In addition, consistent with the long term approach of the 3-year
program, 17 academies (sections) were added and offered in Geometry for students
entering that discipline as 10th graders in fall, 2011, making a total of 39 academy
sections in both algebra and geometry offered to a total of 470 students who completed
the academy, taking both pre and post assessments (see further discussion of
participants below).
This final evaluation report for 2011 will present an overview of the program design and
participants, as well as a discussion of findings from analyses of pre and post-test
measures of student content knowledge and from analyses of student attitudes toward
degree of difficulty of test items. The report will also provide recommendations for
future directions.
7
Program Design Overview
Designed to prepare students entering Algebra I or Geometry in 9th or 10th grade, the
intent of the MAA approach in 2011 was to support students who have previously
struggled in math classes by providing them with an intensive preview of/ introduction to
material they would later see in these algebra and/or geometry courses. Doing so
through approaches that emphasized active learning, collaboration, opportunities to
engage in logical reasoning and creative thinking in order to solve problems, and
rigorous content instruction, was intended to increase student capacity and confidence
in that capacity to succeed in these courses in 9th and/or 10th grade.
The instructional curriculum was originally developed by Dr. Phil Duren, and updated
and significantly expanded by Phil Gonsalves, to include Geometry and Algebra II as
well as new middle school curriculum. Mr. Gonsalves updates the curriculum annually
by getting feedback from teachers in the field.
The MAA program was designed as a three year cycle, with students progressing
through Algebra I preparation, Geometry I, and then Algebra II preparation across each
of the three years. For that reason, students who participated in Algebra I preparation
in 2010, could continue in Year 2 (2011), participating in a Geometry I preparation
academy. In addition, the approach was seen as occurring in overlapping cycles,
because a new cohort was added in 2011 in Algebra I, thus beginning a second cycle of
three years for those students.
The actual summer session occurred over four weeks, five days per week, for three
hours per day (60 hours total). Instructors were primarily drawn from a pool of teachers
who had previously participated in professional development courses offered through
Project ACCLAIM (the Alameda County Collaborative for Learning and Instruction in
Mathematics). These teachers participated in activities that enhanced their capacity to
use such pedagogical approaches as showing multiple algorithms side-by-side,
decomposition, developing visual representations such as bar models, appropriate use
of mathematical syntax and academic language, and other content-focused student
engagement strategies.
MAA teachers were supported by 1-2 student mentors, most often CSUEB mathematics
students, at each host site. These mentors attended academies daily, assisted
instructors as needed, and worked with students in small groups.
Finally, the MAA program is designed to support students throughout the school year.
Study sessions are held prior to key assessments (‘study jams’). Students also have
opportunities for tutoring with student mentors throughout the year.
One additional goal of MAA is to foster relationships between students’ families and
community as well as institutions such as CSU East Bay, and to support families’
capacity to assist their students in pursuing academic and career pathways. To that
end, students and their families attend University Nights prior to the start of summer
8
academies, an evening of information about what is required for going to college, how to
start planning, both academically and financially, and exposure to potential and possible
pathways.
The following discussions of district/sites, student participants, and findings from
analyses of pre/post test results will assess project impact in the context of the summer
academies component of the MAA program.
Project Impact
Project impact can first be assessed in terms of participating districts and students and
will be discussed in the first part of this section of the report.
Project Participants
Twenty-two Algebra academies were offered in 2011 to students from eleven
participating districts in both Alameda and Contra Costa Counties (with one school from
San Benito County). Five of these districts were those originally considered Chevron
districts (Dublin USD, Livermore Valley JUSD, West Contra Costa USD, Mt. Diablo
USD, and Oakland USD). Additional districts included Alameda USD, Fremont USD,
John Swett USD, Hayward USD, Antioch USD, and San Benito High School District.
Seventeen Geometry academies were offered to students from nine of these same
districts (minus Antioch and San Benito districts). The addition of the 17 Geometry
academies represented an expansion of the MAA program, adding additional capacity
to impact students entering ‘gatekeeper’ mathematics courses in high school.
MAA is designed to be a district-based program. Therefore the 39 academies were
hosted across 13 different middle and/or high school sites for Algebra and 10 different
sites for Geometry in the participating districts. Students were recruited by participating
districts (as well as through outreach efforts by CSU East Bay). A variety of local
criteria were used for recruitment, including prior student performance in mathematics,
with the common criterion that students needed to be entering Algebra 1 in 9 th grade or
Geometry in 10th grade in 2011-12. In addition, a priority for the Geometry academies
was placed on encouraging students from Year 1 to continue in the program in Year 2.
Initial analysis of 2011 Geometry enrollment data indicated that over half of the students
enrolled were students continuing from Year 1 (Olkin, 2011 Preliminary Report,
November, 2011).
Table 1 represents the demographic pool from which students were drawn for both the
Algebra and Geometry academies in 2011. Two districts (Antioch, San Benito HS) are
those not offering Geometry academies. As can be seen, while there are variations, a
high number of these districts include a significant percentage of students who receive
free and/or reduced lunch under Federal guidelines, with only two (Dublin, Fremont)
below 20% and the other districts ranging between 25.5% (Livermore VJUSD) and
70.3% (Oakland USD). Free and/or reduced lunch percentage is an indicator used by
schools to identify socioeconomic status. The same can be said in the area of diversity
9
for those districts with significant numbers of often underrepresented students in STEM
pathways (e.g., Latino, African American).
Table 1 Demographics of Districts (2010-2011) with Students Enrolled in Algebra
and/or Geometry Academies (Summer, 2011)
District
Total
Dist
Enr
% Free
& Red.
Lunch
- Dist
% Ethnic/Racial Enrolllment – District*
Latino Hispanic
Af.
Amer
.
White,
Not
Hispanic
Asian
ELL
Filipino.
Alameda
10,494
34.3
13.3
12
30.4
32
8.3
USD
Antioch USD
19,081
57.8
35
22.4
23.3
5
5
Dublin USD
6257
11.2
14.3
6.6
39.7
27.5
8
Fremont
32,607
19.5
15
4.3
19
51
6
USD
Hayward
21,744
62.5
57.3
14.5
8
9
7
USD
John Swett
1,735
45.2
31
20
21
9.5
8.2
USD
Livermore
12,771
25.5
26.3
2.4
56
6
2.8
VJUSD
Mt. Diablo
34,116
39.6
36
5
43
7
4
USD
Oakland
46,584
70.3
40
32
8
13
1
USD
San Benito
HS School
3,072
37.1
63
1
32
1
1.3
Dist
West Contra
29.978
68.7
48
14.4
8.6
8
4
Costa USD
*Note: Racial and/or Ethnic groups with less than 4% across all represented districts are not
represented in this table, primarily for space reasons.
22.3
18.3
9.2
17.4
32.6
16
14
21.4
26
10
33.4
Tables 2 (Algebra academies) and 3 (Geometry academies) below provide a summary
of participants in the 2011 academies. Most districts, with the exception of Fremont,
Antioch, and Oakland (Geometry academy only), hosted two or more academy
‘sections’ (Column 2), taught by one teacher per section (Column 4). These tables offer
a perspective on the fluid nature of enrollment in many of the academies. Column 5 in
each table indicates the total enrollment in a given academy section, by district and
individual teacher/class. Column 6 (Matching Pre/Post Test) represents the number of
students completing the entire academy, taking both a pre and a post math content
assessment. Column 7, on the other hand represents the number of students taking the
pre-test, including those students who enrolled but did not complete the academy,
dropping out of the section at some point before the post-test, while Column 7 shows
the number of students taking the post-test, including those students who enrolled in the
academy sometime after the pre assessment had been given.
10
Table 2 Summary of Participants – Test Data Source - Algebra
Algebra
No.
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
District
2
Alameda USD
Alameda USD
Antioch USD
Dublin USD
Dublin USD
Fremont USD
Hayward USD
Hayward USD
John Swett USD
John Swett USD
Livermore VJUSD
Livermore VJUSD
Mt. Diablo USD
Mt. Diablo USD
Sites Hosting Classes
3
Alameda HS
Alameda HS
Fremont Elementary School
Wells MS
Wells MS
Washington HS
Brenkwitz HS F1, F2, F3, F4
Brenkwitz HS F1, F2, F3, F4
Rodeo Hills ES
Rodeo Hills ES
Livermore HS
Livermore HS
Riverview MS
Riverview MS
15
16
Mt. Diablo USD
Mt. Diablo USD
17
18
19
20
21
22
Oakland USD
Oakland USD
San Benito High School District
San Benito High School District
West Contra Costa USD
West Contra Costa USD
Pleasant Hill MS
Pleasant Hill MS
Colesium College Prep
Academy
Fremon HS
San Benito HS
San Benito HS
Crespi HS
Crespi HS
Total
Teacher
Sending
Students
4
A06
A15
A11
A20
A13
A03
A10
A16
A12
A01
A18
A02
A08
A19
Total
Enrollment
5
34
33
29
36
36
29
24
26
11
11
14
7
16
17
Matching
(Pre&Post)
6
30
23
24
8
9
17
19
18
8
5
8
5
13
13
Pre Test
7
34
32
29
26
28
28
24
26
11
11
14
7
16
17
Post Test
8
30
24
24
18
15
18
19
18
8
5
8
5
13
13
Dropout
Rate
9
0.12
0.28
0.17
0.69
0.68
0.39
0.21
0.31
0.27
0.55
0.43
0.29
0.19
0.24
A22
A05
18
26
9
18
14
22
13
22
0.36
0.18
A07
A21
A14
A04
A09
A17
25
18
17
11
14
19
17
12
17
7
9
6
24
18
17
10
13
17
18
12
17
8
10
8
0.29
0.33
0.00
0.30
0.31
0.65
22
471
295
438
326
0.33
11
Table 3 Summary of Participants – Test Data Source - Geometry
Geometry
No.
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
District
2
Alameda USD
Alameda USD
Dublin USD
Dublin USD
Fremont USD
Hayward USD
Hayward USD
John Swett USD
John Swett USD
Livermore VJUSD
Livermore VJUSD
Mt. Diablo USD
Mt. Diablo USD
Mt. Diablo USD
Oakland USD
West Contra Costa USD
West Contra Costa USD
Total
Sites Hosting Classes
3
Alameda HS
Alameda HS
Wells MS
Wells MS
Washington HS
Brenkwitz HS F1, F2, F3, F4
Brenkwitz HS F1, F2, F3, F4
Rodeo Hills ES
Rodeo Hills ES
Livermore HS
Livermore HS
Riverview MS
Riverview MS
Pleasant Hill MS
Colesium College Prep Academy
Crespi HS
Crespi HS
Teacher
Sending
Students
4
G13
G02
G17
G09
G12
G06
G15
G16
G05
G03
G10
G08
G11
G01
G07
G04
G14
Total
Enrollment
5
19
22
15
16
7
16
15
14
16
14
14
18
13
23
14
5
10
Matching
(Pre&Post)
6
14
15
15
7
4
14
12
9
8
10
12
11
7
18
8
4
7
Pre Test
7
19
22
15
14
7
15
15
14
16
13
14
17
12
19
14
5
10
Post Test
8
14
15
15
9
4
15
12
9
8
11
12
12
8
22
8
4
7
Drop-out
Rate
0.26
0.32
0.00
0.50
0.43
0.07
0.20
0.36
0.50
0.23
0.14
0.35
0.42
0.05
0.43
0.20
0.30
17
251
175
241
185
0.28
12
As shown in Table 2 above, over the course of the Algebra academies, 471 students
were enrolled and served in some way. Students enrolled and completing the pre-test
numbered 438, while 326 completed only the post-test, having enrolled some time
during the four-week academy. Two hundred and ninety-five students, then, completed
all of one four-week Algebra academy, taking both a pre and a post test (with 31
additional students completing the academy but having missed taking the pre-test). No
data is available disaggregating students based on the number of actual days they were
enrolled, so that the number of students taking only the post-test might represent
students who were enrolled for almost the entirety of the academy as well as some who
enrolled much later. The same is true of those who dropped out prior to completing the
post assessment.
Table 3 makes visible that 251 students were enrolled in the Geometry academies, with
175 completing both the pre and post assessments (and ten additional students
completing the academy but having missed taking the pre-test).
Both Tables 2 and 3 show, in Column 8, drop-out rates for the academies (an average
of .33 in Algebra and of .28 in Geometry). It is important to note, however, that
examining these rates from the opposite perspective indicates that the 2011 program
had a 68% retention rate in Algebra and a 72% retention rate in Geometry.
Findings – Pre/Post Math Content Tests
Overview of Overall Pre/Post Math Content Assessment Data in Algebra and Geometry
Students completed a pre and/or post assessment of math content knowledge at the
beginning and end of the four-week summer session. The assessments in both algebra
and geometry (one for each discipline) consisted of 35 multiple choice items. The
algebra assessment, given to students enrolled in the Algebra Academy, included
content such as absolute value, simplifying algebraic expressions, solving algebraic
equations, area, slope, quadratic equations, graphs, and functions. Items were the
same for both pre and post-tests.
Figure 1 shows, broadly, the overall growth in student content knowledge in the MAA
academies across all sections, based on comparison of mean scores (of # correct out of
35 items). In this case, however, data was analyzed for all students taking a pre-test, a
post test, or both a pre and a post test, so results in this analysis are somewhat skewed.
However, raw percentages from these findings indicate growth in algebra academies
from a mean of 40% for all students taking a pre-test to 52% for those students taking a
post test (numbers broken down in Table 2). In addition, findings also indicate growth in
geometry academies from a mean of 37% for all students taking a pre-test to 50% for
those students taking a post test. While this particular figure does not accurately reflect
pre/post findings since it is not disaggregated for those students taking both
assessments, it does point to the possibility of significant overall growth in math content
knowledge in both algebra and geometry academies.
13
Figure 1
MAA 2011 Student Content Knowledge Data:
Mean Growth Across All Classes/Teachers (All
Students Taking Pre and/or Post Tests)
20
15
Pre-Test
10
Post-Test
5
0
Algebra
Geometry
Further understanding of the assessments can be gained by examining Figures 2 and 3
below. As can be seen initially in Figure 2, like Figure 1, the N (sample) for pre-test is
438, while the N (sample) for the post test is 326. The figure shows the distribution of
scores for pre and post-tests, using mean or average number of items answered
correctly, out of 35, based on 438 students who took the pre-test (14.043 average items
correct) and 326 students who took the post-test (18.230 average items correct). The
distribution curve does make visible a general picture of a shift in student performance
on the assessments. As the figure shows, by the post-test, more students, as a group,
were answering more items correctly, with more students getting almost all items
correct than in the pre-test.
By allowing for the average sample size for students who completed the algebra
academy, and using these scores, it is also possible to isolate those students (N=295)
taking both the pre and the post-test. This analysis is able to show growth, for these
students, of 3.831 points (between averages). This analysis supports the finding that
there was, on average, a statistically significant increase in student knowledge of
algebra content.
Figure 3 shows that overall results for the geometry academies are consistent with
findings for the algebra academies. The distribution is more clearly visible in the
geometry data and is supported when the data is adjusted for the 175 students taking
14
Figure 2 Overall Analysis of Pre/Post Test Data – Algebra Academies
Teacher Name: All Teachers
Class:
Algebra
1. Overall Analysis
1.1. Participating Students
Table 1: Descriptive Statistics
Type of Test
N
Mean
Median
Pre-Test
Post-Test
13.000
17.000
438
326
14.043
18.230
St. Deviation
Min (out of
Max (out of
35)
35)
5.492
6.604
0.000
5.000
32.000
34.000
1.2. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.08
Variable
pre-test score
post-test score
0.07
Mean StDev
N
14.04 5.492 438
18.23 6.604 326
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
6
12
18
Data
24
30
1.3. Hypothesis Test
Test Statistic: 14.70
Difference of Averages: 3.831
Sample Size: 295
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
15
Figure 3 Overall Analysis of Pre/Post Test Data – Geometry Academies
Teacher Name: All Teachers
Class:
Geometry
1. Overall Analysis
1.1. Participating Students
Table 1: Descriptive Statistics
Type of Test
N
Mean
Median
Pre-Test
Post-Test
11.000
17.000
241
185
10.975
17.541
St. Deviation
Min (out of
Max (out of
35
35)
4.555
5.678
0.000
1.000
24.000
33.000
1.2. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.09
Variable
pre-test score
post-test score
0.08
Mean StDev
N
10.98 4.555 241
17.54 5.678 185
0.07
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
6
12
18
Data
24
30
1.3. Hypothesis Test
Test Statistic: 15.19
Difference of Averages: 6.120
Sample Size: 175
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
16
both the pre and post-test. In this case, there was a 6.120 difference between the pretest average and the post-test averages, an increase twice as great as that for algebra.
In order to further understand pre and post-test findings in terms of project impact, it is
necessary to examine test data by district and then by class, which will be the focus of
the following sub-section.
Algebra – Pre/Post Test Findings by District and Class
As discussed above, when examined broadly, it appears that all districts represented by
students in the Algebra Academies in 2011 showed growth in mean student content
knowledge. Table 4 presents an analysis of mean pre-test scores and mean post-test
scores for all students, without adjustment for students taking only the pre and post-test.
Table 4 Algebra - Growth in Mean Student Knowledge by District (All Students
Taking Pre and/or Post Test)
School District
Pre-Test – Mean
% Correct
Alameda USD
Mt. Diablo USD
Fremont USD
John Swett USD
San Benito High School District
Oakland USD
Antioch USD
Hayward USD
Dublin USD
Livermore VJUSD
West Contra Costa USD
All (Mean)
39%
44%
49%
35%
36%
33%
41%
39%
46%
50%
31%
40%
Post-Test –
Mean %
Correct
55%
58%
60%
46%
47%
43%
50%
48%
54%
57%
37%
47%
% Point
Increase
16%
14%
11%
11%
11%
10%
9%
9%
8%
7%
6%
10%
Findings presented in Table 4 represent mean numbers converted to percent of total
items correct (out of 35). When analyzed in this way, it is possible to see that Alameda
USD’s students’, for example, mean number of correct answers increased 16%, while
students from West Contra Costa USD, on average, increased by 6% (the difference
between mean percent of items correct).
Table 5 presents a further analysis of districts, disaggregated by class, and adjusted
only for students taking both pre and post-tests (by allowing for sample size for this
group as a test statistic and adjusting differences in mean scores based on sample size
and further identifying whether these differences between averages are statistically
significant enough to show improvement).
This analysis (Table 5) makes visible, for example, that while one class (A20)
representing Dublin USD made only 0.63 average point growth, the second class
showed growth of 4.56 points between averages. The 0.63 differences between
17
averages, while not a decrease, is not considered statistically significant as
improvement over time, but, as the analysis of broad mean scores in Table 4 shows,
Dublin USD students, as a whole, grew 8% overall in pre and post mean scores.
Table 5 Differences in Pre and Post Averages by District and by Class, with
Enrollment Retention Rates by Class - Algebra
District
Teacher
N=#
Taking
Pre
and
Post
30
23
Enrollment
Retention
Rate
Diff Between Mean Scores Statistically
Significant to Indicate Improvement?
A06
A15
Diff.
Betw.
Pre &
Post
Mean
7.18
4.83
Alameda
USD
.88
.72
Yes
Yes
Antioch
USD
A11
2.046
24
.83
Yes
A20
A13
0.63
4.56
8
9
.31
.32
Yes
Fremont
USD
A03
3.882
17
.61
Yes
Hayward
USD
A10
A16
1.53
4.22
19
18
.79
.69
Yes
John Swett
USD
A12
A01
2.75
1.40
8
5
.73
.45
Yes
Livermore
VJUSD
A18
A02
5.75
2.80
8
5
.57
.80
Yes
Mt. Diablo
USD
A08
A19
A22
A05
3.31
2.923
7.22
5.61
13
13
9
18
.81
.76
.64
.82
Yes
Yes
Yes
Yes
Oakland
USD
A07
A21
2.471
3.08
17
12
.71
.67
Yes
Yes
San Benito
HS District
A14
A04
4.790
0.71
17
7
100
.70
Yes
West
Contra
Costa USD
A09
1.13
9
.69
A17
2.83
6
.35
Yes
.68
16 Yes, 6 No
Dublin USD
Total
Average
3.831
No
No
No
No
No
No
18
Figures 4 and 5 present a visual representation of two classes within the same district
and analyses of students’ average growth in algebra content knowledge over time within
each of these classes (academy sections). As shown in Table 2, Oakland Unified
School District (represented in Figures 4 and 5 below) fell exactly in the middle (10%) of
mean percentage growth between pre and post assessments, the same percentage as
that of the average growth across all districts and classes. At the same time, Oakland
students were drawn from a pool with the highest percentage of those receiving free
and reduced lunch (70.3 %), with its percentage of underrepresented students
somewhere near the top of the range represented by all participating districts (e.g., 40%
Latino, 32% African American).
In addition, with a total enrollment across both classes of 43 students, Oakland USD
classes had retention rates of 71% and 67% respectively. Oakland’s retention rates,
particularly that of Teacher A21, are consistent with the average retention rate across all
classes (68%). Finally, as shown in Table 5, both classes showed improvement in
average scores when the sample was adjusted for students taking both the pre and the
post assessment, with students of Teacher A21 (3.08 average point increase) showing
a slightly higher difference between averages than those of Teacher A07 (2.471
average points). Again, the differences between averages for each class are between
.75 and 1.36 points of the average point difference (3.81) across all classes. For these
reasons, Oakland USD has been chosen to serve as an example of further analyses of
each individual class (see Appendix I).
The class distribution of both Oakland USD classes shown in Figures 4 and 5 below,
then, show a shift in the ‘normal curve’ that is consistent with that representing the
distribution of student scores overall in algebra (all districts and classes) across pre and
post assessments, as represented in Figure 2. At the same time, Oakland’s shift in
distribution is representative visually and statistically of shifts in student content
knowledge in most classes when they are analyzed individually. That is, as shown in
Figure 4, for example, more students in Teacher A07’s class were getting more test
items correct in the post assessment than they were in the pre-test and there was a shift
‘to the right’ of the normal curve distribution. In other words, where students started in
the pre-test (minimum number of items correct within the range of correct items) was
lower than the minimum number of items correct received by the lowest scorer on the
post-test. In Teacher A07’s class, that lowest (minimum) number correct was 5 in the
pre-test and 8 in the post-test, while the lowest (minimum) number in Teacher A21’s
class (Figure 5) for the pre-test was 7 and for the post-test, 10. And, as noted, the
maximum score (the highest number correct within the range) was higher for both
classes in the post-test than in the pre-test (i.e., students scoring highest on the test
received more correct answers in the post test than did the highest scorers in the pretest).
Similar individual analyses of all algebra academy classes can be found in Appendix I
and further support the overall findings for the 2011 Algebra Academies discussed in
this section.
19
Figure 4
Teacher:
Class:
School:
District:
A07
Algebra
Colesium College Prep Academy
Oakland USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
Mean
24
18
11.000
14.560
Median
11.000
12.000
St. Deviation
3.648
5.390
Min (out of
Max (out of
35)
35)
5.000
8.000
17.000
27.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.12
Variable
pre-test score
post-test score
0.10
Mean StDev N
11 3.648 24
14.56 5.393 18
Density
0.08
0.06
0.04
0.02
0.00
5
10
15
Data
20
25
3. Hypothesis Test
Test Statistic: 3.30
Difference of Averages: 2.471
Sample Size: 17
P-value: 0.002
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
20
Teacher:
Class:
School:
District:
A21
Algebra
Fremont HS
Oakland USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
Mean
18
12
12.000
15.750
Median
12.000
14.000
St. Deviation
4.201
5.360
Min (out of
Max (out of
35)
35)
7.000
10.00
23.000
28.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.10
Variable
pre-test score
post-test score
Density
0.08
Mean StDev N
12 4.201 18
15.75 5.362 12
0.06
0.04
0.02
0.00
5
10
15
Data
20
25
3. Hypothesis Test
Test Statistic: 3.01
Difference of Averages: 3.08
Sample Size: 12
P-value: 0.006
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
21
Geometry – Pre/Post Test Findings by District and Class
Findings for pre and post assessments of content knowledge for students enrolled in the
Geometry Academies, when the nine participating districts are analyzed individually, are
similar to those presented for the Algebra Academies, as shown in Table 6. When
broad mean averages are analyzed (of all students taking a pre and/or a post-test) by
district, all districts showed a significant increase in mean scores, ranging from 8%
(Oakland USD) to 30% (JohnSwett USD) increase in mean scores (# of items correct
out of 35 total items).
Table 6 Geometry - Growth in Mean Student Knowledge by District (All Students
Taking Pre and/or Post Test)
School District
Pre-Test – Mean
% Correct
John Swett USD
Livermore VJUSD
West Contra Costa USD
Alameda USD
Fremont USD
Hayward USD
Dublin USD
Mt. Diablo USD
Oakland USD
All (Mean)
22%
36%
25%
28%
34%
33%
26%
33%
27%
29%
Post-Test –
Mean %
Correct
52%
61%
50%
51%
57%
54%
45%
46%
35%
50%
% Point
Increase
30%
25%
25%
23%
23%
21%
19%
13%
8%
21%
Findings, when analyzed by district, support initial overall findings in that these broad
percentage point increases between pre and post-assessments are significantly higher
for districts with students enrolled in geometry than for those enrolled in algebra,
although all participating districts in the algebra academies also showed increases in
mean scores between pre and post-tests. This again raises questions about the
potential positive impact of the program, including its long-term approach which
encourages students to continue each year for three years. While no cause/effect
correlation can be claimed, we can at least posit that greater increases in content
knowledge between pre and post assessments in geometry might be potentially linked
to the fact that many of those students enrolled in the geometry academies were
continuing students. Those students potentially had past experience to draw on as
resource in order to understand what counted as participation in the academies. They
had past experience in algebra preparation and a year in this gatekeeper class (Algebra
I). Experience in Algebra I might also be the case for other students who were not
continuing from the previous year. Again, these findings raise new questions about
potential program impact, supporting the concept that continuity is important.
Analyses of districts led, again, to the need to examine individual class (academy
section) and district findings. Table 7 below presents analyses by district and by class
of the differences between pre and post-test averages when sample size is adjusted for
22
only those students taking both the pre and the post-assessment. In this case, while all
classes showed some difference between averages, only two classes had results that
could not be considered statistically significant to indicate improvement.
Table 7 Differences in Pre and Post Averages by District and by Class, with
Enrollment Retention Rates by Class - Geometry
District
Teacher
N=#
Taking
Pre
and
Post
14
15
Enrollment
Retention
Rate
Diff Between Mean Scores Statistically
Significant to Indicate Improvement?
G13
G02
Diff.
Betw.
Pre &
Post
Mean
3.36
6.02
Alameda
USD
.74
.68
Yes
Yes
Dublin USD
G17
G09
2.83
5.51
6
7
1.00
.50
Yes
Fremont
USD
G12
1.08
4
.57
Hayward
USD
G06
G15
4.82
4.89
15
12
.93
.80
Yes
Yes
John Swett
USD
G16
G05
2.23
1.92
9
8
.64
.50
Yes
Yes
Livermore
VJUSD
G03
G10
5.60
9.417
10
12
.77
.86
Yes
Yes
Mt. Diablo
USD
G08
G11
G01
4.09
7.00
4.72
11
7
18
.65
.58
.95
Yes
Yes
Yes
Oakland
USD
G07
2.875
8
.57
Yes
West
Contra
Costa USD
G04
G14
8.75
7.86
4
7
.80
.70
Yes
Yes
.72
15 Yes, 2 No
Total
Average
6.120
No
No
When analyzed in this way, the average difference between mean scores (pre and posttest) is 6.12 (3.81 for Algebra Academies). At the same time, the Geometry Academies,
on average, had a 72% retention rate (68% retention rate in algebra). This set of
analyses of individual classes adjusted for those students taking both the pre and the
post test (N=175) again leads to potential questions about the impact of encouraging
students to participate in the Math Achievement Academies across multiple years.
23
The two classes drawing students from Alameda USD (Teacher G13 and Teacher G02)
have been selected to make visible a more micro analytic layer of individual classes
within districts. In this case, the district selected again fell somewhere in the middle of
the nine districts in terms of broad mean percentage score increase between pre and
post assessments of geometry content knowledge, as shown in Table 6 (23%).
Alameda USD falls somewhere in the mid-range of Federal free and/or reduced lunch
rates (34%), and in the lower range, compared to other districts, in its percentage of
students identified as Latino or African American (13% and 12% respectively).
In addition, Alameda classes had retention rates of 74% (Teacher G13) and 68%
(Teacher G02), each within 2 and 4 percentage points of the overall average retention
rates across all classes (68%), as shown in Table 7. Finally, while Teacher G02’s class
had approximately twice as large a point difference between pre/post averages (3.36)
as Teacher G13 (6.02), both classes showed significant improvement.
As shown in Figures 6 and 7 below, both classes from Alameda USD show a significant
shift in class distribution between pre and post assessments. Class G13 shows, for
example, a significant shift in the numbers of students falling within a narrower range of
scores (8 points between the lowest and the highest score on the post test, in contrast
to 14 points between lowest and highest on the pre-test). In addition, the ‘starting point’
or minimum score in the range rose from 1 to 11 on the post-test. While students on
average in class G13 made this growth, the distribution is narrower in that the range
was narrower than that of Class G02.
As Figure 7 shows, Class G02 had more students receiving more correct answers in the
post test. At the same time, their ‘starting point’ (the lowest number of correct answers
– minimum) on the post test was higher than that of Class G13. That minimum number
of correct items on the post-test (13) for Class G02 was significantly higher, too, than its
lowest score on the pre-test (0).
The findings from Alameda USD classes are consistent with those shown in the overall
distribution for all classes, all teachers, shown in Figure 2.
Similar individual analyses of all geometry academy classes can be found in Appendix II
and further support the overall findings for the 2011 Geometry Academies discussed in
this section.
24
Figure 6
Teacher:
Class:
School:
District:
G13
Geometry
Alameda High School
Alameda Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
Mean
19
14
9.789
15.286
Median
10.000
15.500
St. Deviation
3.473
2.555
Min (out of
Max (out of
35)
35)
1
11
15
19
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.16
Test Ty pe
PostTest
PreTest
0.14
Mean StDev N
15.29 2.555 14
9.789 3.473 19
Density
0.12
0.10
0.08
0.06
0.04
0.02
0.00
4
8
12
Test Scores
16
20
Geometry
3. Hypothesis Test
Average Difference: 3.36
T-statistic: 4.66
Sample Size: 14
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
25
Figure 7
Teacher:
G02
Class:
Geometry
School: Alameda High School
District: Alameda Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
Mean
22
15
9.82
20.07
Median
11.00
19.00
St. Deviation
5.88
5.50
Min (out of
Max (out of
35)
35)
0
13
22
33
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.08
Test Ty pe
Post Test
Pre Test
0.07
Mean StDev N
20.07 5.496 15
9.818 5.885 22
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
8
16
Test Scores
24
32
Geometry
3. Hypothesis Test
Average Difference: 6.02
T-statistic: 7.29
Sample Size: 15
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
26
Findings – Pre and Post Student Self-Assessment of Item Difficulty – Algebra and
Geometry
As students took pre and post-tests of content knowledge, they were also asked to
assess the degree of difficulty of each of the 35 items as they came to it (selfassessment, opinion). Students were asked to make these assessments in both
algebra (Algebra Academy pre and post-test) and geometry (Geometry Academy pre
and post-test). Findings from analyses of pre and post student self-assessment of item
difficulty in both algebra and geometry are discussed in this section.
Students were asked, on both the pre and the post test, to rate each item, as they
completed it, according to their assessment of its degree of difficulty. Students were
asked to give their opinion as to whether they considered the item to be simple,
challenging, or difficult. It was hypothesized that the number of items rated as
challenging or difficult should decrease between the pre and the post test and the
number rated simple should increase, thus serving as an indicator of increased student
confidence in their capacity as well as increase in that capacity itself to solve algebraic
or geometric problems.
A sample item similar to those of the Geometry assessment is shown below. In
addition, the place for the student to give his/her opinion about the degree of difficulty of
the item is also shown. Items on the Algebra assessment are similarly formatted.
7
What is the distance between points A (3, -7)
and B (9, -2)?
A)
61
B)
117
C)
61
D)
25
Simple
Challenging
Difficult
27
In Analysis I, the average degree of difficulty indicated by students on both the pre and
the post-test was calculated and contrasted for all 35 items for both Algebra and
Geometry academies. Whether or not students answered items correctly is not
correlated with their indication of difficulty in this analysis. However, as shown in both
Figures 8 (Algebra) and 9 (Geometry), below, while there is some variation among
items, the overall average reported difficulty for each item decreased between the pre
and the post-test.
Figure 8 Average Student-Reported Difficulty across 35 Questions in Algebra –
Pre-Test vs. Post-Test
Average Difficulty for Algebra
3.00
2.50
2.00
1.50
1.00
0.50
0.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
PRE
POST
Figure 9 Average Student-Reported Difficulty across 35 Questions in Geometry –
Pre-Test vs. Post-Test
Average Difficulty for Geometry
3.00
2.50
2.00
1.50
1.00
0.50
0.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
PRE
POST
28
A second layer of analysis, presented in Figures 10 (Algebra) and 11 (Geometry),
makes visible even more clearly, on an item by item basis, the shifts in student
perception of difficulty between the pre and the post test. In this case, each item is
broken down according to percentage of students reporting that item as simple (blue),
challenging (green), or difficult (red).
Figures 10 and 11, first, clearly support the initial hypothesis that students would
increasingly indicate items as simple by the post-test and decreasingly indicate items as
challenging and/or difficult. As indicated by the blue portions of the lines for each item
in Figure 10, the percentage of students indicating that they saw an item as ‘simple’ on
the Algebra pre-test, doubled, and often tripled, if not more, by the post test. In
Geometry (Figure 11), the same is true for most items. At the same time, the numbers
of students indicating those same items as challenging decreased significantly, as
indicated by the shortened portions of each line colored green on the post-test, in both
algebra and geometry.
In addition, as both figures show, some items increased significantly in the average
number of students reporting them as simple, such that they also decreased
significantly in numbers identifying those same items as either challenging or difficult
(e.g., items 1 and 35 in algebra; items 1 and 29 in geometry). For many items,
however, analysis showed, as represented in both Figure 10 and Figure 11, that the
average number of items marked as ‘difficult’ (red) in contrast to ‘challenging’ (green),
did not decrease significantly. This might be explained in several ways. Since,
together, items marked challenging and difficult decreased in the post-test for algebra
and geometry, it might be argued that some students had difficulty distinguishing
between the two choices. It also might be argued that more students taking only the
post-test (see Tables 2 and 3 above), selecting level of difficulty for the first time, may
have perceived more items as difficult.
The findings discussed in this section make visible a progressive shift between pre and
post-tests in both Algebra and Geometry Academies in students self-reporting from high
levels of difficulty of test items to increasingly perceiving items as ‘simple’, or less
‘challenging’ or ‘difficult’.
29
Figure 10 Shifts in Student Assessment of Difficulty Between Pre and Post Tests - Algebra
Pre
Post
34
34
32
32
30
30
28
28
26
26
24
24
22
22
20
20
18
18
16
16
13
13
11
11
9
9
7
7
5
5
3
3
1
1
0%
20%
Simple
40%
60%
Challenging
80%
Difficult
100%
0%
20%
Simple
40%
60%
Challenging
80%
100%
Difficult
30
Figure 11 Shifts in Student Assessment of Difficulty Between Pre and Post Tests – Geometry
Pre
Post
34
34
32
32
30
30
28
28
26
26
24
24
22
22
20
20
18
18
16
16
13
13
11
11
9
9
7
7
5
5
3
3
1
1
0%
20%
Simple
40%
60%
Challenging
80%
Difficult
100%
0%
20%
Simple
40%
60%
Challenging
80%
100%
Difficult
31
Concluding Remarks
Overall findings from analyses of assessments of student content knowledge following
participation in the summer Math Achievement Academies indicates a significant level
of success in terms of student growth. Findings support the conclusion that the
program is having a positive impact on average student growth in content knowledge,
as evidenced by the overall growth in algebra. Even if students attend only one year of
summer academy, algebra results support the summer academy approach that
emphasizes preparation rather than remediation. Additionally significant are the
differences in growth in geometry academies. It is recommended that additional
attention be paid to this finding in Year 3, in order to assess student participation,
opportunities for learning, and the impact of the program on those students who
attended academies for multiple years. The findings from this analysis of the
introduction of the geometry academies in 2011 supports the positive potential and
consequences for student learning of a long-term approach to student preparation for
entering gatekeeper math courses.
In addition, analyses of student self-assessment of item degree of difficulty indicated a
strong shift in how students perceived test items in both algebra and geometry. The
percentage of students indicating that an item was simple significantly increased, while
indications that an item was challenging or difficult significantly decreased between the
pre and post-tests. It is recommended that correlation between self-reported degree of
difficulty and whether an item is solved correctly be analyzed. It would also be helpful to
have test items assessed by test makers as to their perceptions of problem level of
difficulty in order to contrast with student perceptions and student actual answers
(correct or incorrect).
An added recommendation is related to analyses contrasting individual classes. This
information might be more useful in future if it were accompanied by some analyses of
classes in action – that is, what was made available to students and how students took
up opportunities for learning afforded them. This may be a direction to be explored in
future.
32
Appendices
33
Appendix I
Algebra - Student Content Knowledge Test Analyses by Class
(Listed Alphabetically by District)
34
Teacher:
Class:
School:
District:
A06
Algebra
Alameda HS
Alameda USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
34
30
Median
13.500
21.000
Mean
14.353
20.170
St. Deviation
5.493
6.260
Min
3.000
10.000
Max
25.000
31.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.08
Variable
Pre-Score
Post-Score
0.07
Mean StDev N
14.35 5.493 34
20.17 6.265 30
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
4
8
12
16
20
Data
24
28
32
3. Hypothesis Test
Test Statistic: 5.333
Difference of Averages: 7.18
Sample Size: 30
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
35
Teacher:
Class:
School:
District:
A15
Algebra
Alameda HS
Alameda USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
32
24
Median
12.500
20.00
Mean
12.813
18.580
St. Deviation
4.987
5.580
Min
0.000
8.000
Max
23.000
30.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.09
Variable
Pre-Score
Post-Score
0.08
Mean StDev N
12.81 4.987 32
18.58 5.579 24
0.07
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
8
16
Data
24
32
3. Hypothesis Test
Test Statistic: 4.91
Difference of Averages: 4.83
Sample Size: 23
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
36
Teacher:
Class:
School:
District:
A11
Algebra
Fremont Elementary
Antioch USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
29
24
Mean
14.241
17.625
Median
15.000
17.000
St. Deviation
3.757
4.126
Min
6
10
Max
20
23
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.12
Test Ty pe
Post Test
Pre Test
0.10
Mean StDev N
17.63 4.126 24
14.24 3.757 29
Density
0.08
0.06
0.04
0.02
0.00
8
12
16
Test Scores
20
24
Algebra
3. Hypothesis Test
Average Difference: 2.046
T-statistic: 4.44
Sample Size: 24
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
37
Teacher:
Class:
School:
District:
A20
Algebra
Wells MS
Dublin USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
26
18
Median
17.000
19.500
Mean
17.654
18.940
St. Deviation
5.012
6.810
Min
10.000
6.000
Max
26.000
29.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
Variable
Pre-Score
Post-Score
0.08
0.07
Mean StDev N
17.65 5.012 26
18.94 6.812 18
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
5
10
15
20
Data
25
30
35
3. Hypothesis Test
Test Statistic: 0.41
Difference of Averages: 0.63
Sample Size: 8
P-value: 0.347
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
38
Teacher:
Class:
School:
District:
A13
Algebra
Wells MS
Dublin USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
28
15
Median
12.50
20.00
Mean
14.61
18.80
St. Deviation
7.10
6.43
Min
0.00
9.00
Max
29.00
26.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.07
Variable
Pre-Score
Post-Score
0.06
Mean StDev N
14.61 7.099 28
18.8 6.428 15
Density
0.05
0.04
0.03
0.02
0.01
0.00
0
8
16
Data
24
32
3. Hypothesis Test
Average Difference: 4.56
T-statistic: 2.00
Sample Size: 9
P-value: 0.040
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
39
Teacher:
Class:
School:
District:
A03
Algebra
Washington HS
Fremont USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
28
18
Median
16.50
20.50
Mean
17.00
21.11
St. Deviation
6.54
6.96
Min
6.00
5.00
Max
29.00
31.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
Variable
Pre-Score
Post-Score
0.06
0.05
Mean StDev N
17 6.543 28
21.11 6.961 18
Density
0.04
0.03
0.02
0.01
0.00
5
10
15
20
Data
25
30
35
3. Hypothesis Test
Average Difference: 3.882
T-statistic: 5.62
Sample Size: 17
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
40
Teacher:
Class:
School:
District:
A10
Algebra
Brenkwitz HS F1, F2, F3, F4
Hayward USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
24
19
Mean
15.750
17.950
Median
15.000
18.00
St. Deviation
4.542
7.830
Min
7.000
7.000
Max
27.000
32.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.09
Variable
Pre-Score
Post-Score
0.08
Mean StDev N
15.75 4.542 24
17.95 7.835 19
0.07
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
8
16
Data
24
32
3. Hypothesis Test
Average Difference: 1.53
T-statistic: 1.28
Sample Size: 19
P-value: 0.109
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
41
Teacher:
Class:
School:
District:
A16
Algebra
Brenkwitz HS F1, F2, F3, F4
Hayward USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
26
18
Median
10.500
14.50
Mean
11.423
15.940
St. Deviation
4.941
7.620
Min
1.000
7.000
Max
23.000
32.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.09
Variable
Pre-Score
Post-Score
0.08
Mean StDev N
11.42 4.941 26
15.94 7.619 18
0.07
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
5
10
15
Data
20
25
30
3. Hypothesis Test
Average Difference: 4.22
T-statistic: 3.64
Sample Size: 18
P-value: 0.001
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
42
Teacher:
Class:
School:
District:
A12
Algebra
Rodeo Hills ES
John Swett USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
11
8
Median
12.000
16.000
Mean
12.000
15.130
St. Deviation
2.098
3.910
Min
8.000
9.000
Max
15.000
21.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.20
Variable
Pre-Score
Post-Score
Mean StDev N
12 2.098 11
15.13 3.907 8
Density
0.15
0.10
0.05
0.00
8
12
16
Data
20
24
3. Hypothesis Test
Average Difference: 2.75
T-statistic: 2.62
Sample Size: 8
P-value: 0.017
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
43
Teacher:
Class:
School:
District:
A01
Algebra
Rodeo Hills ES
John Swett USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
11
5
Mean
12.27
17.40
Median
10.00
17.00
St. Deviation
4.41
2.61
Min
8
15
Max
19
21
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.16
Variable
Pre-Score
Post-Score
0.14
Mean StDev N
12.27 4.407 11
17.4 2.608 5
Density
0.12
0.10
0.08
0.06
0.04
0.02
0.00
4
8
12
Data
16
20
3. Hypothesis Test
Average Difference: 1.40
T-statistic: 1.03
Sample Size: 5
P-value: 0.181
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
44
Teacher:
Class:
School:
District:
A18
Algebra
Livermore HS
Livermore VJUSD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
14
8
Median
14.50
18.50
Mean
15.86
19.63
St. Deviation
6.42
7.69
Min
7.00
7.00
Max
25.00
30.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.07
Variable
Pre-Score
Post-Score
0.06
Mean StDev N
15.86 6.419 14
19.63 7.689 8
Density
0.05
0.04
0.03
0.02
0.01
0.00
5
10
15
20
Data
25
30
35
3. Hypothesis Test
Average Difference: 5.75
T-statistic: 2.77
Sample Size: 8
P-value: 0.014
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
45
Teacher:
Class:
School:
District:
A02
Algebra
Livermore HS
Livermore VJUSD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
7
5
Median
16.00
21.00
Mean
18.86
20.40
St. Deviation
6.77
8.41
Min
12.00
7.00
Max
32.00
28.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.06
Variable
Pre-Score
Post-Score
0.05
Mean StDev N
18.86 6.768 7
20.4 8.414 5
Density
0.04
0.03
0.02
0.01
0.00
5
10
15
20
Data
25
30
35
40
3. Hypothesis Test
Average Difference: 2.80
T-statistic: 0.98
Sample Size: 5
P-value: 0.191
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
46
Teacher:
Class:
School:
District:
A08
Algebra
Riverview MS
Mt. Diablo USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
16
13
Median
12.50
15.00
Mean
12.94
16.77
St. Deviation
4.60
6.60
Min
5.00
9.00
Max
25.00
32.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.09
Variable
Pre-Score
Post-Score
0.08
Mean StDev N
12.94 4.597 16
16.77 6.597 13
0.07
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
5
10
15
20
25
30
Data
3. Hypothesis Test
Average Difference: 3.31
T-statistic: 2.64
Sample Size: 13
P-value: 0.011
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
47
Teacher:
Class:
School:
District:
A19
Algebra
Riverview MS
Mt. Diablo USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
17
13
Median
12.00
17.00
Mean
13.18
16.46
St. Deviation
4.61
6.51
Min
5.00
8.00
Max
22.00
27.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.09
Variable
Pre-Score
Post-Score
0.08
Mean StDev N
13.18 4.613 17
16.46 6.514 13
0.07
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
5
10
15
Data
20
25
30
3. Hypothesis Test
Average Difference: 2.923
T-statistic: 3.56
Sample Size: 13
P-value: 0.002
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
48
Teacher:
Class:
School:
District:
A22
Algebra
Pleasant Hill MS
Mt. Diablo USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
14
13
Median
20.00
30.00
Mean
18.64
26.62
St. Deviation
7.43
7.09
Min
5.00
11.00
Max
30.00
34.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.06
Variable
Pre-Score
Post-Score
0.05
Mean StDev N
18.64 7.428 14
26.62 7.089 13
Density
0.04
0.03
0.02
0.01
0.00
5
10
15
20
25
Data
30
35
40
3. Hypothesis Test
Average Difference: 7.22
T-statistic: 4.40
Sample Size: 9
P-value: 0.001
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
49
Teacher: A05
Class:
Algebra
School: Pleasant Hill MS
District: Mt. Diablo USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
22
22
Median
17.00
21.00
Mean
16.27
21.73
St. Deviation
5.41
6.81
Min
10.00
8.00
Max
28.00
31.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of Pre-Score, Post-Score
Normal
0.08
Variable
Pre-Score
Post-Score
0.07
Mean StDev N
16.27 5.409 22
21.73 6.812 22
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
5
10
15
20
Data
25
30
35
3. Hypothesis Test
Average Difference: 5.61
T-statistic: 4.93
Sample Size: 18
P-value: 0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
50
Teacher:
Class:
School:
District:
A07
Algebra
Colesium College Prep Academy
Oakland USD
4. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
24
18
Median
11.000
12.000
Mean
11.000
14.560
St. Deviation
3.648
5.390
Min
5.000
8.000
Max
17.000
27.000
5. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.12
Variable
pre-test score
post-test score
0.10
Mean StDev N
11 3.648 24
14.56 5.393 18
Density
0.08
0.06
0.04
0.02
0.00
5
10
15
Data
20
25
6. Hypothesis Test
Test Statistic: 3.30
Difference of Averages: 2.471
Sample Size: 17
P-value: 0.002
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
51
Teacher:
Class:
School:
District:
A21
Algebra
Fremont HS
Oakland USD
4. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
18
12
Median
12.000
14.000
Mean
12.000
15.750
St. Deviation
4.201
5.360
Min
7.000
10.00
Max
23.000
28.000
5. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.10
Variable
pre-test score
post-test score
Density
0.08
Mean StDev N
12 4.201 18
15.75 5.362 12
0.06
0.04
0.02
0.00
5
10
15
Data
20
25
6. Hypothesis Test
Test Statistic: 3.01
Difference of Averages: 3.08
Sample Size: 12
P-value: 0.006
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
52
Teacher:
Class:
School:
District:
A14
Algebra
San Benito High School
San Benito High School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
17
17
Mean
12.53
18.88
Median
12.00
17.00
St. Deviation
5.68
4.24
Min
4
12
Max
24
26
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
Test Ty pe
PostTest
PreTest
0.09
0.08
Mean StDev N
18.88 4.241 17
12.53 5.680 17
Density
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
5
10
15
Test Scores
20
25
Algebra
3. Hypothesis Test
Average Difference: 4.790
T-statistic: 7.10
Sample Size: 17
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
53
Teacher:
Class:
School:
District:
A04
Algebra
San Benito High School
San Benito High School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
10
8
Median
13.00
13.50
Mean
13.10
14.13
St. Deviation
3.143
3.310
Min
9
9
Max
18
19
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.14
Test Ty pe
Post Test
Pre Test
0.12
Mean StDev N
14.13 3.314 8
13.1 3.143 10
Density
0.10
0.08
0.06
0.04
0.02
0.00
8
12
16
Test Scores
20
Algebra
3. Hypothesis Test
Average Difference: 0.71
T-statistic: 0.55
Sample Size: 7
P-value: 0.302
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
54
Teacher: A09
Class:
Algebra
School: Crespi MS
District: West Contra Costa USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
13
10
Median
10.000
11.500
Mean
10.308
11.900
St. Deviation
3.401
4.040
Min
5.000
7.000
Max
17.000
18.000
2. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.12
Variable
pre-test score
post-test score
0.10
Mean StDev N
10.31 3.401 13
11.9 4.040 10
Density
0.08
0.06
0.04
0.02
0.00
2.5
5.0
7.5
10.0
12.5
Data
15.0
17.5
20.0
3. Hypothesis Test
Test Statistic: 1.13
Difference of Averages: 1.67
Sample Size: 9
P-value: 0.145
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
55
Teacher:
Class:
School:
District:
A17
Algebra
Crespi MS
West Contra Costa USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
17
8
Median
12.000
13.500
Mean
11.294
13.880
St. Deviation
2.823
4.120
Min
7.00
9.00
Max
17.00
20.00
2. Class Distribution
Figure 1: Normal Curve
Histogram of pre-test score, post-test score
Normal
0.16
Variable
pre-test score
post-test score
0.14
Mean StDev N
11.29 2.823 17
13.88 4.121 8
Density
0.12
0.10
0.08
0.06
0.04
0.02
0.00
4
8
12
16
20
Data
3. Hypothesis Test
Test Statistic: 1.99
Difference of Averages: 2.83
Sample Size: 6
P-value: 0.052
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
56
Appendix II
Geometry - Student Content Knowledge Test Analyses by Class
(Listed Alphabetically by District)
57
Teacher:
Class:
School:
District:
G13
Geometry
Alameda High School
Alameda Unified School District
4. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
19
14
Median
10.000
15.500
Mean
9.789
15.286
St. Deviation
3.473
2.555
Min
1
11
Max
15
19
5. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.16
Test Ty pe
PostTest
PreTest
0.14
Mean StDev N
15.29 2.555 14
9.789 3.473 19
Density
0.12
0.10
0.08
0.06
0.04
0.02
0.00
4
8
12
Test Scores
16
20
Geometry
6. Hypothesis Test
Average Difference: 3.36
T-statistic: 4.66
Sample Size: 14
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
58
Teacher:
G02
Class:
Geometry
School: Alameda High School
District: Alameda Unified School District
4. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
22
15
Median
11.00
19.00
Mean
9.82
20.07
St. Deviation
5.88
5.50
Min
0
13
Max
22
33
5. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.08
Test Ty pe
Post Test
Pre Test
0.07
Mean StDev N
20.07 5.496 15
9.818 5.885 22
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
8
16
Test Scores
24
32
Geometry
6. Hypothesis Test
Average Difference: 6.02
T-statistic: 7.29
Sample Size: 15
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
59
Teacher:
Class:
School:
District:
G17
Geometry
Wells Middle School
Dublin Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
15
15
Mean
13.933
16.670
Median
14.000
16.000
St. Deviation
3.127
5.000
Min
9
7
Max
21
27
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.14
Test Ty pe
PostTest
PreTest
0.12
Mean StDev N
16.67 4.995 15
13.93 3.127 15
Density
0.10
0.08
0.06
0.04
0.02
0.00
5
10
15
20
Test Scores
25
Geometry
3. Hypothesis Test
Average Difference: -0.14
T-statistic: 1.68
Sample Size: 15
P-value: 0.058
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
60
Teacher:
Class:
School:
District:
G09
Geometry
Wells Middle School
Dublin Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
14
9
Mean
4.500
14.556
Median
5.000
15.000
St. Deviation
3.276
2.603
Min
0
11
Max
11
18
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.16
Test Ty pe
PostTest
PreTest
0.14
Mean StDev N
14.56 2.603 9
4.5 3.276 14
Density
0.12
0.10
0.08
0.06
0.04
0.02
0.00
0
4
8
12
Test Scores
16
20
Geometry
3. Hypothesis Test
Average Difference: 5.51
T-statistic: 4.89
Sample Size: 7
P-value: 0.001
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
61
Teacher:
Class:
School:
District:
G12
Geometry
Washington High School
Fremont Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
7
4
Median
12
20
Mean
12
20
St. Deviation
2.236
6.580
Min
8
13
Max
15
27
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.20
Test Ty pe
PostTest
PreTest
Mean StDev N
20 6.583 4
12 2.236 7
Density
0.15
0.10
0.05
0.00
8
12
16
20
24
Test Scores
28
32
Geometry
3. Hypothesis Test
Average Difference: -1.08
T-statistic: 2.03
Sample Size: 4
P-value: 0.068
Output summary:
We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
62
Teacher:
Class:
School:
District:
G06
Geometry
Brenkwitz High School F1, F2, F3, F4
Hayward Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
15
15
Mean
10.867
18.070
Median
11.000
19.000
St. Deviation
3.523
4.990
Min
5
8
Max
17
26
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.12
Test Ty pe
PostTest
PreTest
0.10
Mean StDev N
18.07 4.992 15
10.87 3.523 15
Density
0.08
0.06
0.04
0.02
0.00
5
10
15
20
Test Scores
25
30
Geometry
3. Hypothesis Test
Average Difference: 4.82
T-statistic: 5.32
Sample Size: 15
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
63
Teacher:
Class:
School:
District:
G15
Geometry
Brenkwitz High School F1, F2, F3, F4
Hayward Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
15
12
Median
12.00
20.00
Mean
12.20
19.83
St. Deviation
3.299
6.090
Min
12
20
Max
6
10
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
Test Ty pe
PostTest
PreTest
0.12
0.10
Mean StDev N
19.83 6.088 12
12.2 3.299 15
Density
0.08
0.06
0.04
0.02
0.00
5
10
15
20
Test Scores
25
30
Geometry
3. Hypothesis Test
Average Difference: 4.89
T-statistic: 4.79
Sample Size: 12
P-value: <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
64
Teacher:
Class:
School:
District:
G16
Geometry
Rodeo Hills Elementary School
John Swett Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
14
9
Median
14.50
20.00
Mean
13.50
19.22
St. Deviation
4.69
5.17
Min
8.75
13.5
Max
19
25
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.09
Test Ty pe
PostTest
PreTest
0.08
0.07
Mean StDev N
19.22 5.167 9
13.5 4.686 14
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
5
10
15
20
Test Scores
25
30
Geometry
3. Hypothesis Test
Average Difference: 2.23
T-statistic: 3.83
Sample Size: 9
P-value: 0.002
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
65
Teacher:
Class:
School:
District:
G05
Geometry
Rodeo Hills Elementary School
John Swett Unified School District
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
16
8
Mean
11.625
17.000
Median
11.000
16.500
St. Deviation
2.918
4.630
Min
7
12
Max
16
27
2. Class Distribution
Figure 1: Normal Curve
Class Distribution
Normal
0.14
Test Ty pe
PostTest
PreTest
0.12
Mean StDev N
17 4.629 8
11.63 2.918 16
Density
0.10
0.08
0.06
0.04
0.02
0.00
5
10
15
20
Test Scores
25
Geometry
3. Hypothesis Test
Average Difference: 1.92
T-statistic: 3.24
Sample Size: 8
P-value: 0.007
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
66
Teacher:
Class:
School:
District:
G03
Geometry
Livermore HS
Livermore VJUSD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
13
11
Median
11.00
20.00
Mean
13.54
20.64
St. Deviation
6.12
5.57
Min
6.00
14.00
Max
24.00
31.00
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.08
Test Ty pe
Post Test
Pre Test
0.07
Mean StDev N
20.64 5.573 11
13.54 6.118 13
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
5
10
15
20
Stack Score
25
30
3. Hypothesis Test
Average Difference: 5.60
T-Value : 2.31
Sample Size: 10
P-Value : 0.023
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
67
Teacher:
Class:
School:
District:
G10
Geometry
Livermore HS
Livermore VJUSD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
14
12
Median
11.500
20.00
Mean
11.929
21.830
St. Deviation
3.668
4.760
Min
7.000
16.000
Max
19.000
28.000
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.12
Test Ty pe
Post Test
Pre Test
0.10
Mean StDev N
21.83 4.764 12
11.93 3.668 14
Density
0.08
0.06
0.04
0.02
0.00
5
10
15
20
Stack Score
25
30
3. Hypothesis Test
Average Difference: 9.417
T-Value : 10.27
Sample Size: 12
P-Value : <0.0005
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
68
Teacher:
Class:
School:
District:
G08
Geometry
Riverview HS
Mt. Diablo USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
17
12
Median
13.00
17.00
Mean
12.71
16.42
St. Deviation
5.32
7.89
Min
0.00
1.00
Max
21.00
27.00
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.08
Test Ty pe
Post Test
Pre Test
0.07
Mean StDev N
16.42 7.891 12
12.71 5.324 17
Density
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
8
16
Stack Score
24
32
3. Hypothesis Test
Average Difference: 4.09
T-Value: 2.44
Sample Size: 11
P-Value: 0.017
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
69
Teacher:
Class:
School:
District:
G11
Geometry
Riverview HS
Mt. Diablo USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
12
8
Median
11.00
17.00
Mean
10.92
16.50
St. Deviation
4.21
7.33
Min
6.00
7.00
Max
17.00
30.00
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.10
Test Ty pe
Post Test
Pre Test
Density
0.08
Mean StDev N
16.5 7.329 8
10.92 4.209 12
0.06
0.04
0.02
0.00
0
5
10
15
20
Stack Score
25
30
3. Hypothesis Test
Average Difference: 7.00
T-Value: 3.92
Sample Size: 7
P-Value: 0.004
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
70
Teacher:
G01
Class:
Geometry
School: Pleasant Hill MS
District: Mt. Diablo USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
19
22
Mean
10.895
15.68
Median
12.00
14.50
St. Deviation
3.526
6.03
Min
2.00
6.00
Max
17.000
27.00
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.12
Test Ty pe
Post Test
Pre Test
0.10
Mean StDev N
15.68 6.027 22
10.89 3.526 19
Density
0.08
0.06
0.04
0.02
0.00
6
12
18
Stack Score
24
30
3. Hypothesis Test
Average Difference: 4.72
T-Value: 3.44
Sample Size: 18
P-Value: 0.002
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
71
Teacher:
Class:
School:
District:
G07
Geometry
Colesium College Prep Academy
Oakland USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
14
8
Mean
9.57
12.25
Median
9.00
11.50
St. Deviation
4.11
3.01
Min
1.00
7.00
Max
19.00
16.00
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.14
Test Ty pe
Post Test
Pre Test
0.12
Mean StDev N
12.25 3.012 8
9.571 4.108 14
Density
0.10
0.08
0.06
0.04
0.02
0.00
0
4
8
12
Stack Score
16
3. Hypothesis Test
Average Difference: 2.875
T-Value: 3.37
Sample Size: 8
P-Value: 0.006
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
72
Teacher:
Class:
School:
District:
G04
Geometry
Crespi MS
West Contra Costa USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
5
4
Median
10.00
18.00
Mean
9.60
18.25
St. Deviation
2.70
3.30
Min
5.00
15.00
Max
12.00
22.00
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.16
Test Ty pe
Post Test
Pre Test
0.14
Mean StDev N
18.25 3.304 4
9.6 2.702 5
Density
0.12
0.10
0.08
0.06
0.04
0.02
0.00
5
10
15
Stack Score
20
25
3. Hypothesis Test
Average Difference: 8.75
T-Value: 2.83
Sample Size: 4
P-Value: 0.033
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
73
Teacher:
Class:
School:
District:
G14
Geometry
Crespi MS
West Contra Costa USD
1. Participating Students
Table 1: Descriptive Statistics
Type of Test
Pre-Test
Post-Test
N
10
7
Median
9.00
18.00
Mean
8.70
16.71
St. Deviation
3.43
6.52
Min
3.00
6.00
Max
13.00
25.00
2. Class Distribution
Figure 1: Normal Curve
Class Distributions
Normal
0.12
Test Ty pe
Post Test
Pre Test
0.10
Mean StDev N
16.71 6.525 7
8.7 3.433 10
Density
0.08
0.06
0.04
0.02
0.00
4
8
12
16
20
Stack Score
24
28
32
3. Hypothesis Test
Average Difference: 7.86
T-Value: 4.67
Sample Size: 7
P-Value: 0.002
Output summary:
We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores.
74