Comparing College Students’ Performance in Traditional and
Reform Pre-Calculus and Calculus Courses: An Evaluation Report
Kabba E. Colley, Ed.D.
Introduction
The Long Island Consortium for Interconnected Learning in Quantitative Disciplines (LICIL) is a consortium of ten colleges in Long Island, NY, which was founded in 1996 to improve and
enhance the teaching and learning of mathematics and mathematics-based disciplines at the college
level. The consortium is funded by the National Science Foundation under an initiative called Mathematical Sciences and Their Applications Throughout the Curriculum. The member institutions are
Dowling College; Long Island University C.W. Post Campus; Nassau Community College; New
York Institute of Technology; St. Joseph’s College, Suffolk Community College; SUNY, Farmingdale; SUNY, Old Westbury; SUNY, Stony Brook and York College, CUNY (Long Island Consortium, 1997). Subsequently, the U.S. Merchant Marine Academy was added to the Consortium.
With support from LICIL, a group of faculty from the mathematics department at New York
Institute of Technology has been experimenting with new ways of teaching undergraduate mathematics at the precalculus and calculus level. In the Fall of 1999, they implemented a study to investigate the relationship between mathematics teaching and students’ mathematical performance in a
pre-calculus course. Four sections of pre-calculus were offered, two of which were taught using a
traditional approach and the other two using a reform approach based on mathematical modeling.
While the core topics covered in both precalculus groups were the same, the emphasis and content
were quite different. The traditional course stressed routine algebraic manipulations to improve
student skills. The reform/modeling course stressed conceptual understanding of the mathematical
ideas, problem-solving, and realistic applications. In the reform/modeling course, the algebraic manipulations arose only in the context of problem solving, not as long lists of drill-and-skill problems. Students learned by “applying the mathematical ideas and methods to real-world data that
they personally obtained on subjects of interest to them” (Gordon, 2000). The traditional course
Page 1
was mainly lecture-based and the textbook used emphasized drill and practice. In both teaching approaches, students were required to use the TI-86 or equivalent graphing calculator.
Prior to the pre-calculus course, a placement test was administered to students in both
groups.
The group entering the two reform/modeling pre-calculus sections had lower scores (a
mean of 12.47 out of 20) than those entering the two traditional sections (a mean of 13.58 out of
20); however, this difference in means is not statistically significant.
Under the LICIL grant, a separate pre/post attitudinal survey was conducted by the LICIL
external evaluator, Leo Gafney on the first and the last day of class in all sections of the precalculus
course in Fall, 1999. The results are included in Appendix II.
For the present study, student performance was analyzed on a series of ten common questions (see Appendix I) on the final examinations for both precalculus groups. These ten questions
were all primarily manipulative in nature, actually giving an advantage to the students from the traditional sections since their courses had emphasized precisely those algebraic skills, while the reform/modeling sections had emphasized conceptual understanding, problem solving, and realistic
applications.
In the Spring of 2000, a follow-up study was conducted on student performance and retention in Calculus I based on whether the students had been through a traditional pre-calculus course
or the reform/modeling pre-calculus course. Not all students who take precalculus are required to
take Calculus I and some do not go on to Calculus I the immediately following semester. Thus, only 13 students who had taken the reform/modeling course during the Fall 1999 study were enrolled
in Calculus I during Spring, 2000; 37 of the students in Calculus I had come through a traditional
precalculus course. The students were not separated into two groups based on their precalculus
background; rather, both groups were taught by the same professor in two sections of the same calculus course and had the same learning experiences. The method of teaching employed was a mixture of traditional and reform elements.
During the follow-up study in calculus, students’ performance on quizzes, class tests and the
final examination were compared, as well as student success rates, retention rates, and persistence in
the course.
The purpose of this evaluation report is to present the methods, findings, and policy implications of the Fall, 1999 pre-calculus study and the Spring, 2000 calculus follow-up study. The report
Page 2
is divided into three sections. The first section describes the methodology of the study. The second
section presents the results of the pre-calculus and calculus study. The final section summarizes the
results from the two studies and discusses their policy implications.
Summary of Findings
I. Precalculus

Placement test results: The group entering the traditional precalculus sections had higher
average scores on the departmental placement test than those entering the reform/modeling
sections.

The 10 common questions on the two precalculus finals were all primarily manipulative in
nature, giving an advantage to the students from the traditional sections since their courses had
emphasized precisely those skills while the reform/modeling sections had emphasized conceptual understanding, problem solving, and realistic applications.

Students in the reform/modeling sections outperformed those in the traditional sections on
7 of the 10 common questions on the precalculus final exams.

Success rates (percent who passed) in precalculus were comparable for the two groups.

Results of the attitudinal studies in precalculus demonstrated that students in the re-
form/modeling sections:
(1) developed better attitudes toward the usefulness and importance of mathematics as it applies to other disciplines;
(2) gained more confidence in their ability to do mathematics; and
(3) had a higher appreciation for the importance of technology in problem-solving.
II. Calculus

In the follow-up calculus course, students coming from the reform/modeling sec-
tions consistently outperformed students coming from a traditional precalculus background:
(1) on all quizzes,
(2) on all class tests, and
 In calculus,
(3) onstudents
the finalcoming
exam from the reform/modeling sections demonstrated signifi.
cantly higher persistence and retention rates in the course. 92.3% of the students with
a reform/ modeling background took the final examination in calculus while 64.9% of
those with a traditional background took the final examination in calculus.
Page 3
 Success rates in passing calculus were significantly higher among those students who
had been in the reform/modeling sections (76.9%) compared to those who had been in
Methods
Guiding Research Question
The guiding research question for this evaluation was stated as follows: “Is student mathematical performance in a pre-calculus and a follow-up calculus course related to the teaching approach used to teach the precalculus course?” The outcome variable for this evaluation was students’ overall mathematical performance in a pre-calculus and in a follow-up calculus course. The
predictor variables for this evaluation were the teaching approach used to teach the pre-calculus
course, students’ gender and race. A related question was: “Is student attitude, persistence and retention in a precalculus and a follow-up calculus course related to the teaching approach used to teach
the precalculus course?”
Evaluation Design
The evaluation design employed in this study can be described as a pretest-posttest control
group design without random assignment. The groups of students and professors involved in this
study were already formed, without randomization. However, both groups of students in precalculus were similar, except for the teaching approach they received in their pre-calculus courses. It is
important to note that students had no idea in advance, when they registered for pre-calculus, that
some of the sections would be reform/modeling and others traditional. Thus, their choice of section
was basically random. Moreover, during the first week of the semester, the students in the reform/modeling sections were told that the classes were going to be very different from the traditional approach and they were given the option of transferring to one of the traditional sections, if they
so chose. Not one of the students opted out. This comparable offer was never made to the students
in the traditional sections.
Page 4
Description of Sample
Sixty-four students participated in the pre-calculus study and of these, 23 were females and
41 were males. In addition, 19 students were categorized as Asian, 6 as Blacks, 4 as Hispanic, 27 as
Whites and 8 as “Others.” In the follow-up calculus study, 50 students participated. Out of these,
15 were females and 35 were males. The racial breakdown for this group was as follows: 15 Asian,
6 Blacks, 6 Hispanics, 21 Whites, and 2 categorized as “Others”. In the calculus study, 13 students
were from the reform/modeling background in precalculus and 37 were from a traditional background in precalculus. Table 1 shows the characteristics of the students who participated in the precalculus and calculus follow-up study.
Table 1: Characteristics of students participating in the pre-calculus and calculus study.
Pre-Calculus Study
Variable
Females
Males
Asian
Blacks
Hispanic
White
Other
Reform Pre-Calculus
Traditional Pre-Calculus
Pre-Calculus Section I
Pre-Calculus Section II
Pre-Calculus Section III
Pre-Calculus Section IV
Calculus Study
N
23
41
19
6
4
27
8
37
27
18
9
19
18
Variable
Females
Males
Asian
Black
Hispanic
White
Other
Reform
Traditional
N
15
35
15
6
6
21
2
13
37
Data Collection
The data used in this evaluation were as follows: students’ scores on the Fall, 1999 mathematics placement test; students’ scores on ten common questions on the Fall 1999 pre-calculus
course final examination; and students’ scores on seven quizzes, three class-tests and a final examination on the Spring 2000 calculus course.
Page 5
Data Analysis
Analysis of the data used for this evaluation was conducted using the JMP Software (SAS
Institute, 1995). The JMP Software is a powerful statistical package that allows users to conduct a
wide variety of statistical analysis including univarate, bivariate and multivariate analysis. In addition, JMP has the capability to fit complex models and provide users with a variety of statistical
tests to choose from. In this evaluation, the JMP Software was used first to conduct exploratory data analysis (Tukey, 1977) and summarize the data. To determine if students’ mathematics performance differed by teaching approach, the JMP Software was used to conduct Multivariate Analysis
of Variance (MANOVA). According to Hamer et al. (1998), MANOVA is an appropriate analytical
procedure when data consists of two or more categorical predictors with two or more continuous
response variables considered concurrently. Other than determining if the two groups differed by
teaching approach, models were also fitted to investigate the effects of section, race and gender on
the outcome variable. All the model tests conducted were set at the  = 0.05 level of significance.
Results from the Pre-Calculus Study
A placement test was administered to all students before the start of classes to determine
what level math course they were ready for. The test solely measures purely algebraic skills. For
those students who placed into precalculus, students in the reform/modeling approach had a mean
score of 12.47, with a standard deviation of 3.93, while those in the traditional precalculus classes
earned a mean score of 13.58, with a standard deviation of 3.36 (see Figure 1). The standard deviations for both scores are very similar indicating that there was very little variation in the placement
test scores of both groups. To determine if the two mean scores were significantly different, a
means/ANOVA t-Test was conducted and the results indicate that, while the students in the traditional classes had better placement scores, the two means were not significantly different: F Ratio =
1.3426, p-value = 0.2512.
The ten common questions on the final exams for both groups in precalculus were worth a
total of 66 points. Students in the reform/modeling approach had a mean score of 49.69 with a
standard deviation of 9.32, while those in the traditional approach scored a mean of 43.63 with a
standard deviation of 12.03. This means that students in the reform/modeling approach did better
than those in the traditional approach, although there is much variation in the scores of the latter
Page 6
group. A means/ANOVA t-Test indicated that the two means were significantly different (F Ratio
= 5.1572, p- value = 0.0266). Thus, the two groups do differ in their performance on the common,
algebraic manipulation questions on the final examinations. (See Figure 1.)
Traditional
Reform
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15
Mean -- Placement Test
Traditional
Reform
0
10
20
30
40
50
60
Mean -- Final Exam
Figure 1: Comparison of mean placement test and mean final examination scores by
teaching approach in a pre-calculus course.
In addition to the above combined score, student performance on each of the ten common
questions was analyzed separately. The students from the reform/modeling classes out-performed
the students from the traditional group in seven of the ten questions: question #1, #5, #6, #7, #8, #9
and #10 (see Figure 2). The ten common questions, as well as their individual point allotments, are
given in Appendix I.
Page 7
Traditional
Ref orm
7
6
5
4
3
2
1
0
8
9
10
Mean(Ques1)
Traditional
Ref orm
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15
Mean(Ques2)
Traditional
Ref orm
0
2
1
3
Mean(Ques3)
Figure 2: Comparison of mean scores on ten questions in a pre-calculus final
examination by teaching approach.
Page 8
Traditional
Ref orm
.0
.5
1.0
1.5
2.0
Mean(Ques4)
Traditional
Ref orm
.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
Mean(Ques5)
Traditional
Ref orm
0
1
2
3
4
3
4
Mean(Ques6)
Traditional
Ref orm
0
1
2
Mean(Ques7)
Figure 2 Continued: Comparison of mean scores on ten questions in a
pre-calculus final examination by teaching approach.
Page 9
Traditional
Ref orm
0
6
5
4
3
2
1
Mean(Ques8)
Traditional
Ref orm
3
2
1
0
Mean(Ques9)
Traditional
Ref orm
.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
Mean(Ques10)
Figure 2 Continued: Comparison of mean scores on ten questions in a
pre-calculus final examination by teaching approach.
To determine whether significant differences exist between the scores earned by the reform/modeling group and the traditional group on these ten questions, a MANOVA model was developed with questions 1-10 as outcomes and teaching approach as a predictor. The results indicate
that the two groups were significantly different on questions # 1, # 4 and # 5 (see Table 2).
As mentioned previously, Appendix I contains all ten of the common questions from the final exams. The three questions where there were significant differences, and discussions regarding
each of them, are as follows.
Page 10
Table 2: Results of MANOVA model with questions 1-10 as outcomes and teaching approach
as a predictor.
Item
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Reform/modeling
Traditional
Mean
9.14
10.13
2.5
1
7.20
3.23
3.46
4.15
2.80
6.08
Mean
6.33
11.09
2.91
1.81
4.44
2.70
3.22
3.63
2.37
5.11
Std.
1.38
3.58
1.66
0.97
1.77
0.97
0.98
1.03
2.21
2.64
Std.
3.71
2.87
1.23
0.56
2.69
1.23
1.05
1.66
1.92
3.04
F Ratio
17.8202
1.3156
1.1639
15.2715
24.4292
3.6470
0.8597
2.3867
0.6484
1.8510
p-value
0.0001
0.2558
0.2848
0.0002
0.0001
0.0608
0.3574
0.1275
0.4237
0.1786
Question #1 Brookville College enrolled 2546 students in 1996 and 2702 students in 1998. Assume
that enrollment follows a linear growth pattern.
(a) Write a linear equation that gives the enrollment in terms of the year t (let t = 0 represent
1996).
(b) If the trend continues, what will the enrollment be in the year 2016?
(c) What is the slope of the line you found in part (a)?
(d) Explain, using an English sentence, the meaning of the slope here.
(e) If the trend continues, when will the enrollment reach 3500 students?
Out of a total of 10 points, the students in the reform/modeling sections scored a mean of 9.14 with
standard deviation of 1.38 while the students in the traditional sections scored a mean of 6.33 with
standard deviation 3.71. A means/ANOVA t-test indicates that the two means were significantly
different (F-ratio = 17.8202, p-value = 0.0001).
Page 11
Question #4 Given h(x) = 4(x2 – 17)15, find two functions f(x) and g(x) such that h(x) = f(g(x)).
Out of a total of 2 points, the students in the reform/modeling sections scored a mean of 1 with
standard deviation of 0.97 while the students in the traditional sections scored a mean of 1.81 with
standard deviation 0.56. A means/ANOVA t-test indicates that the two means were significantly
different (F-ratio = 15.2715, p-value = 0.0002).
Question #5 The population of Peru was 24 million in 1995 and has been growing at an annual
rate of 2.1%.
(a) Write a function for the population P(t) after t years. (Let t = 0 represent 1995.)
(b) According to this model, what will the population be in the year 2000?
(c) Use any method to determine when the population will reach 30 million.
(Note that the above question was given to the students in the reform/modeling sections; a comparable problem asking identical questions dealing with balance in a bank account was given to the
students in the traditional sections.)
Out of a total of 8 points, the students in the reform/modeling sections scored a mean of 7.2
with standard deviation of 1.77 while the students in the traditional sections scored a mean of 4.44
with standard deviation 2.69. A means/ANOVA t-test indicates that the two means were significantly different (F-ratio = 24.4292, p-value = 0.0001).
Perhaps the most striking difference in the two groups is a comparison of the nature of the
responses to part (d) of Question #1 asking the students to interpret the meaning of the slope of the
line. The responses from the students in the reform/modeling sections are shown in Table 3a; those
from the students in the traditional sections are shown in Table 3b. Virtually every student in the
reform/modeling sections gave a meaningful response indicating an understanding of the significance of the slope of a line. In comparison, only about one-third of the students in the traditional
sections were able to provide a meaningful response indicating an understanding of what the slope
represents. Many either left the question out altogether (indicated by NA in the Table) or simply
rephrased the algebraic formula for slope in words.
Page 12
Table 3a: Responses on Meaning of Slope in Reform/Modeling Sections
1. Every year, enrollment increases by 78 students.
2. The yearly increase of enrollment is 78 per year.
3. The enrollment increases by 78 students every year.
4. Every t year it will increase by that #.
5. The slope is the growth in enrollment.
6. The enrollment of students goes by 2/15 x a year
7. The slope is the amount of students that enroll per year.
8. Every year student population increases by 78 students.
9. The population of students increases by 78 every 2 years.
10. The yearly increase of students is 78.
11. The enrollment increases 78 students every one years.
12. The yearly increasing in students enrollment is 78.
13. It means that the number of students enrolling per year is 78.
14. The increasing in students is 78 per year, starting at 1996.
15. Yearly increases in students enrollment is 78 every year.
16. Every year there will be an increase of 78 students.
17. Slope is the increase in the # of enrolled students per year at Brookville college.
18. Every year increase 78 students.
19. This means that for every year the number of students increases by 78.
20. The slope means that for every additional year the number of students increase by 78.
21. For every year that passes, the student number enrolled increases 78 on the previous year.
22. As each year goes by, the # of enrolled students goes up by 78.
23. This means that every year the number of enrolled students goes up by 78 students.
24. The slope means that the number of students enrolled in Brookville college increases by 78.
25. Every year after 1996, 78 more students will enroll at Brookville college.
26. Number of students enrolled increases by 78 each year.
27. This means that for every year, the amount of enrolled students increase by 78.
28. Student enrollment increases by an average of 78 per year.
29. For every year that goes by, enrollment raises by 78 students.
Page 13
30. That means every year the # of students enrolled increases by 2,780 students.
31. For every year that passes there will be 78 more students enrolled at Brookville college.
32. The slope means that every year, the enrollment of students increases by 78 people.
33. Brookville college enrolled students increasing by 0.06127.
34. Every two years that passes the number of students which is increasing the enrollment into
Brookville College is 156.
35. This means that the college will enroll .0128 more students each year.
36. By every two year increase the amount of students goes up by 78 students.
37. The number of students enrolled increases by 78 every 2 years.
Table 3b: Responses on Meaning of Slope in Traditional Sections
1. The slope indicates the average increase per yr.
2. The difference in (y2 – y1)/ (x2 – x1)
3. Every year there is an increase of 78 students.
4. * n/a
5. Slope would be a constant increase or decreasing of a line. So if you would enroll 1 person a
year the constant would be one and so would the slope.
6. It is the change in students per year starting at 1996.
7. Enrollments per year.
8. The slope is the amount of the new students the school gets each year.
9. The point in which the # of students is increasing.
10. The meaning of the slope is the amount that is gained in years and students in a given
amount of time.
11. The ratio of students to the number of years.
12. Difference of the y’s over the x’s.
13. Since it is positive it increases.
14. On a graph, for every point you move to the right on the x-axis. You move up 78 points on the
y-axis.
15. The slope in this equation means the students enrolled in 1996. Y = MX + B .
Page 14
16. The amount of students that enroll within a period of time.
17. Every year the enrollment increases by 78 students.
18. The slope here is 78 which means for each unit of time, (1 year) there are 78 more students enrolled.
19. *n/a
20. *n/a
21. *n/a
22. *n/a
23. The change in the x-coordinates over the change in the y-coordinates.
24. This is the rise in the number of students.
25. The slope is the average amount of years it takes to get 156 more students enrolled in the
school.
26. Its how many times a year it increases.
27. The slope is the increase of students per year.
Note that both groups had comparable ability to calculate the slope of a line. However, unless explicit attention is devoted to emphasizing the conceptual understanding of what the slope
means, it appears that the majority of students are not able to create viable interpretations on their
own and so are not able to apply the mathematics to realistic situations. Moreover, if they are unable to do so with a concept as simple as the slope of a line, which they have undoubtedly encountered in previous mathematics courses, it is unlikely that they will be able to create such interpretations and connections on their own for more sophisticated mathematical concepts. This observation
likely has very significant implications regarding the teaching of most other topics and methods in
mathematics.
Demographic Analysis in Precalculus
Although this evaluation focused primarily on the relationship between teaching approach in
precalculus and students’ mathematical performance, the analysis also included looking to see if differences existed by course section, race and gender. The results are presented in Figure 3 and 4, and
Page 15
Table 4. It can be seen in Figure 3, that Sections One and Three in the reform/modeling approach
differed from Sections Four and Two in the traditional approach on student scores on the
16
Mean -- Placement Test
14
12
10
8
6
4
2
0
One
Three
Four
Reform
Two
One
Three
Four
Traditional
Two
Section within Background
70
Mean -- Final Exam
60
50
40
30
20
10
0
One
Three
Four
Reform
Two
One
Three
Four
Traditional
Two
Section within Background
Figure 3: Comparison of mean placement test scores and mean final examination
scores by teaching approach and section in a pre-calculus course.
Page 16
placement test. Section Four in the traditional approach achieved the highest placement test scores.
On the final examination, Sections One and Three in the reform/modeling approach and Section
Two in the traditional approach seemed to be at the same level, while Section Four seemed to lose
ground. This suggests that there is more variation in what the students master within the traditional
sections. It could also be due to the effect of sample size. One of the sections in the traditional approach (Section Two) had only half the number of students compared to the other sections.
Table 4: Comparison of mean placement test and final examination scores by race and gender
in a pre-calculus course.
Gender
Race
N
F
F
F
F
F
M
M
M
M
M
Asian
Black
Hispanic
Other
White
Asian
Black
Hispanic
Other
White
12
2
1
2
6
7
4
3
6
21
Mean
(PT)
13.2
13
.
14
11.2
12.4
11.7
16
14
12.8
Std.
(PT)
3.74
5.66
.
1.41
2.86
4.12
2.52
7.0
3.69
3.67
Mean
(FE)
50.0
46.5
37
40
55.8
50
43.6
53.2
46.1
45.7
Std.
(FE)
12.25
20.50
.
2.12
9.12
5.02
10.50
5.69
13.55
11.32
To determine if the differences in sections were significant, a MANOVA model was fitted
with all ten questions on the final examination as outcome and section as a predictor. The results
from the whole model indicated that section had an effect on students’ mathematical performance
(F = 2.7766, p-value = 0.0001). A review of the individual t-Test on each question by section revealed that only question # 1, #4 and # 5 were significant (question # 1, F = 8.7032 and p-value
0.0001; question # 4, F = 6.4297 and p-value 0.0008; question # 5, F = 9.8978 and p-value 0.0001).
Page 17
Mean -- Placement Test
20
18
16
14
12
10
8
6
4
Female
White
Other
Hispanic
Blac k
Asian
White
Other
Hispanic
Blac k
0
Asian
2
Male
Rac e within Gender
60
Mean -- Final Exam
50
40
30
20
Female
White
Other
Hispanic
Blac k
Asian
White
Other
Hispanic
Blac k
0
Asian
10
Male
Rac e within Gender
Figure 4: Comparison of mean placement test and mean final examination scores by
race and gender in pre-calculus.
Page 18
An inspection of Figure 4 and Table 4 reveals some interesting information about the performance of students on the placement test and final examination by gender and race. For instance,
on the placement test among males, Hispanic males had the highest mean score of 16.0 with a
standard deviation of 7.0, while Black males gained the lowest mean score of 11.7 with a standard
deviation of 2.52. Among females, those who categorized themselves as ‘Other’ had the highest
mean score of 14.0 with a standard deviation of 1.41, while White females scored the lowest mean
of 11.2 and a standard deviation of 2.86. When male students were compared on their performance
on the final examination, Hispanic males again had the highest score of 53.2, standard deviation
5.69, while Black males scored the least (43.6, standard deviation 10.50). In terms of female students’ performance on the final examination, White females had the highest score of 55.8 (standard
deviation 9.12), while Hispanic females registered the lowest mean score of 37.0 with no standard
deviation.
To determine if there were any gender or race effects on students’ performance on the
placement and final examination, a MANOVA model with gender and race as main effect and as
interaction terms was fitted. The results from the whole model indicated that gender and race and
their interaction term had no effect on students’ mathematics performance (F = 1.1091, p-value =
0.2619).
Results from the Calculus Study
During the following semester, Spring 2000, the study continued in two sections of Calculus
I, Math 170. Students with both the reform/modeling and traditional precalculus backgrounds were
enrolled in the same sections of calculus. Their performance was monitored throughout the semester and compared based on their precalculus background. In particular, this calculus study was
based on the analysis of students’ scores on seven quizzes, three class-tests and a final examination.
As can be seen graphically in Figure 5, the reform/modeling group consistently outperformed the traditional group in all the seven quizzes administered during the Spring, 2000 semester.
The quizzes on which the students with the traditional background came closest to matching the
students from the reform/modeling approach are the quizzes that were primarily algebraic in content
(see quiz # 4 and # 6).
Page 19
A ve ra ge Quiz Grades
Performance on Quizzes
20
Reform
16
12
Traditional
8
1
2
3
4
5
6
7
Quiz Number
Figure 5: Comparison of mean score on seven quizzes by teaching approach in a calculus
course (N = 13 for reform/modeling and N = 37 for traditional).
A close examination of the quiz data reveals a very striking pattern in terms of student persistence in the calculus course depending on which precalculus approach the students had had. Figure 6a diagrams whether each of the students with the traditional background took or did not take
each of the quizzes in calculus. Each row represents an individual student and each column represents a particular quiz (#1 through #7). In particular, each darkened box represents a quiz that a
student took and each blank box represents a missed quiz. Similarly, Figure 6b illustrates the same
information for the students with the reform/modeling background. From an inspection of the two
figures, it is clear that only one student with the reform/modeling background stopped attending calculus. It is equally evident that the persistence levels among the students with a traditional background were considerably lower and that a large component of this group gave up on calculus and
stopped attending.
Page 20
Persistence on Quizzes
Traditional Group
Figure 6a: Persistence on Calculus Quizzes: Students in Traditional Group
Page 21
Persistence on Quizzes
Reform/Modeling
Group
Figure 6b: Persistence on Calculus Quizzes: Students in Reform/Modeling Group
In addition to comparing the reform/modeling and traditional groups on the seven quizzes,
further analysis and examination of the data on the three class-tests administered in calculus during
the Spring 2000 semester was also conducted. The reform/modeling group consistently outperformed the traditional group in all three class-tests (see Figure 7).
Specifically, in the first calculus test, the students coming from the traditional approach had
a mean score of 64.36, with a standard deviation of 19.23. In comparison, the students coming from
the reform/modeling sections had a mean of 90.62, with a standard deviation of 8.96. The standard
deviation for the group in the traditional approach indicated that there was a large variation in test
scores. The minimum score for the traditional group was 32, while their maximum score was 97.
The scores for the reform/modeling group ranged from a minimum of 75 to a maximum of 103. It
is interesting to note that the mean score for the traditional group (64.36) was well below the minimum score of the reform/modeling group (75).
Page 22
To determine if the two mean scores were significantly different, a means/ANOVA t-Test
was conducted and the results indicated that the two means were significantly different, F Ratio =
22.0989, p-value = 0.0001.
In the second test, students from the traditional approach scored a mean 59.58 with a standard deviation of 21.82, while those coming from the reform/modeling sections scored a mean of
77.42 with a standard deviation of 14.26. A means/ANOVA t-Test indicated that the two means
were significantly different (F Ratio =-6.8285, p-value = 0.0125).
The results of the third class test were similar to the second, with students in the traditional
group scoring a mean grade of 62.35, with a standard deviation of 17.31, and those in the reform/modeling group scoring a mean of 76.08, with a standard deviation of 18.13. The F Ratio for
this difference of means is 4.8085 and the associated p-value is p = 0.0355, thus indicating that the
difference in the two means is statistically significant.
There are two facts that are worth relating. First, the students in the reform/modeling group
scored significantly higher than their counterparts in the traditional group on all three class-tests, as
seen in Figure 7.
Average Test Grades
Performance on Class Tests
Reform
100.0
80.0
60.0
40.0
Traditional
20.0
0.0
1
2
3
Class-Test Number
Reform
Traditional
Test 1
90.62 (8.96)
64.36 (19.23)
Test 2
77.42 (14.26)
59.58 (21.82)
Test 3
76.08 (18.13)
62.35 (17.31)
Figure 7: Comparison of mean scores on three tests in a calculus course (N = 13 for
reform/modeling and N = 37 for traditional; standard deviations are in parenthesis).
Page 23
Second, a substantial number of the students with a traditional background (13 out of 37)
stopped attending class and so did not take the later tests. These students were the weakest students
in the group and, had they taken further tests, they would undoubtedly have lowered the mean for
their group and the resulting differences would have been larger still. Thus, the apparent convergence of the two graphs in Figure 7 is somewhat misleading, since it does not take into account the
fact that so many of the weaker students are not reflected. Note that only one student from the reform/modeling group stopped attending the calculus course; he was an above-average student who
had to drop the course for personal reasons.
In fact, among the 13 students in calculus who had the reform/modeling approach in precalculus, 12 (or 92.3%) completed the calculus course and took the final exam. Among the 37 students
who had the traditional approach in precalculus, 24 (or 64.9%) completed the calculus course and
took the final exam. See Figure 8.
64.9%
2
Traditional
Reform
92.3%
1
0
20
40
60
80
100
Figure 8: Percentage of Students Taking the Final Exam in Calculus
Analysis of student scores on the final exam in calculus showed that students coming from
the reform/modeling approach scored a mean of 69.8 with a standard deviation of 26.21. In comparison, the students coming from a traditional approach scored a mean of 55.8 with a standard deviation of 23.09. See Figure 9. The scores for both groups varied widely, ranging from a minimum
Page 24
of 11 to a maximum of 103 for the reform/modeling group and ranging from a minimum of 5 to a
maximum of 97 for the traditional group. There was no significant difference between the two
mean scores (F Ratio = 2.6897, p-value = 0.1102).
55.8
2
Traditional
Reform
1
69.8
0
10
20
30
40
50
60
70
80
Figure 9: Average Scores on Final Exam in Calculus
Again, it is worth noting that a substantial number of the worst-performing students with a
traditional background in precalculus did not take the final exam; had they done so, there would
likely have been a larger discrepancy in the difference of means on the final exam.
Furthermore, there were extreme differences in the percentages of students from the two
precalculus backgrounds who received passing grades in calculus. In particular, among those having the reform/modeling background, 76.9% received a passing grade in calculus while 42.1% of
2
42.1%
Traditional
1
76.9%
Reform
0
20
40
Page 25
60
Figure 10: Percentage of Students Passing
Calculus
80
100
those students having a traditional precalculus background received a passing grade in calculus. See
Figure 10.
Further discussion of persistence and retention in the calculus study will be taken up in the
discussion section.
Demographic Analysis in Calculus
In addition to comparing student performance, retention, and success rates in calculus based
on their precalculus background, a variety of analyses were performed to see if any differences in
performance exist based on demographic characteristics such as gender and race.
Analysis of students’ performance on the seven calculus quizzes by gender and race was
conducted as a way to explore the data and to see how these variables might be affected by the two
teaching approaches. The results are displayed in Figure 11. An examination of this figure shows
that Asian males outperformed their female counterparts in 5 of the 7 quizzes, (exceptions were
quiz # 2 and #5). Black males did better than Black females in 5 of the 7 quizzes (exceptions were
quiz # 1 and # 6). Hispanic males outperformed their female counterparts in all quizzes, although
there were no female Hispanics to compare with in quiz # 6, and # 7. Male students categorized as
“Others” outperformed everyone in quiz # 2 and did relatively well in all the quizzes. There were
no female students categorized as “Others.” White males outperformed White females in 5 of the 7
quizzes (exceptions were quiz # 1 and # 7). However, it is important to note that the gap between
White male and White female quiz scores is very small. Overall, Black and Hispanic students
showed the highest quiz scores except in quiz # 2 and # 7.
Page 26
Figure 11: Comparison of mean quiz scores by gender and race in calculus.
Page 27
Figure 11 continued: Comparison of mean quiz scores by gender and race in calculus.
When students’ performance on the three tests were compared by gender and race, patterns
similar to the ones in the seven quizzes were again found. See Figure 12. For instance, Asian
males outperformed their female counterparts in all tests except in test # 3. Black males outperformed Black females. Hispanic males scored higher than their female counterparts in test # 1. In
test # 2 and # 3, there were no Hispanic females to compare with. Male students categorized as
Page 28
“Others” registered the highest score in all three tests, although, there were no females categorized
as “Others.”
Figure 12: Comparison of mean scores on three tests by gender and race in calculus.
Page 29
White females did better than White males in test # 1, but lost ground in test # 2 and # 3. Again, the
gap between the two was small.
Figure 13: Comparison of mean final examination scores by gender and race in calculus.
A close inspection of Figure 13 indicates that students’ performance in the final examination in the calculus course differed by gender and race. Specifically, Asian males performed better
than Asian females, while White males and White females appear to be on the same performance
level. Black males outperformed every gender/race group. There were no Black females, Hispanic
females and female students categorized as “Others” to compare with.
To determine if the differences in gender by race were significant, a MANOVA model was
fitted with the seven quizzes, three tests and final examination scores as outcome and gender, race
and their interaction terms as a predictor. The results from the whole model indicated that gender/race differences observed in performance in calculus were not significant (F = 0.2755 p-value =
0.9954).
Summary and Policy Implications
Page 30
The guiding research question in this study was: “Is student mathematical performance in a
pre-calculus and a follow-up calculus course related to the teaching approach used to teach the precalculus course?” A related question was: “Is student attitude, persistence and retention in a precalculus and a follow-up calculus course related to the teaching approach used to teach the precalculus
course?” The findings from the pre-calculus and calculus follow-up study demonstrated that students mathematical performance was related to teaching approach. Students in the reform/modeling
approach in both the pre-calculus and calculus outperformed the students in the traditional approach
in a number of assessments. This is consistent with earlier findings in a comparable study conducted by Gordon (Gordon, 1995). In that study, the same instructor taught both the reform/
model-
ing precalculus course and the traditional precalculus course during the same semester.
The results of the present performance, retention, and persistence study are also worth relating to the results of the attitudinal survey conducted by (Gafney, 1999; also see Appendix II) of the
same students while they were taking the reform/modeling or the traditional precalculus course during Fall, 1999. The students in the reform/modeling sections developed considerably better attitudes toward the usefulness and importance of mathematics as it applies to other disciplines, gained
more confidence in their ability to do mathematics, and had a higher appreciation for the importance
of technology in problem-solving.
Specifically, in the pre-calculus study, students in the reform/modeling group out-performed
the traditional group in seven of the ten common questions on the final exam. Significant differences existed in the scores earned on three questions, as previously discussed.
In the calculus follow-up study, the reform/modeling group consistently outperformed the
traditional group in all seven quizzes, all three class-tests, and on the final exam administered during the Spring, 2000 semester. However, it is important to note that in both the traditional and reform/modeling group, there were students who missed one or more quizzes as well as one or more
class-tests and the final exam; this was much more so with the traditional group. The issue of missing scores raises two important questions for this study. One question is: What are the possible explanations for the fact that so many students missed quizzes and tests? The second question is: How
does this impact the results of the study? There could be several reasons for students in the calculus
course to miss quizzes and tests, and these could range from personal to school related reasons. For
instance, one possible explanation is that students drop out because they may not be able to cope
Page 31
with the demands of course work or because of work schedule. It may also be due to unanticipated
difficulties or a student’s realizing that he or she is already failing and deciding to stop attending
and withdraw from that course.
The answer to the second question depends to some extent on how the findings from this
study will be utilized. It is anticipated that the findings from this study will be used to improve
mathematics teaching and learning at NYIT and similar institutions in LICIL. Any attempt to generalize the findings from this study to the population of colleges and universities in the US must be
done with caution because the sample size and the lack of a statistically selected random sample for
this study makes it inappropriate for generalization. Nevertheless, given the limitations of the study
design, the findings could be used to inform the teaching practices of mathematics faculty at NYIT.
It can also be used to improve curriculum development in mathematics and related disciplines and
student advising in pre-calculus and calculus. In addition, the findings from this study could be
used to plan new studies that focus on some of the issues raised in the pre-calculus and calculus
study using a larger sample of students and faculty that could be generalizable to the larger population.
One of the unintended outcomes of this study is gender and race differences in mathematics
performance. In both the pre-calculus and calculus study, male students were observed to outperformed females, even though this was not found to be significant. This raises some questions about
whether reform/modeling and traditional approaches to teaching pre-calculus and calculus are equal
in terms of benefits to different genders. In other words, they may differ in terms of student performance, but may be similar in that males tend to perform better than females.
Another interesting finding from this study is that Black and Hispanic students appears to be
performing better and in some cases even outperformed every gender/race group in the reform/modeling approach. Although the numbers of Blacks and Hispanics in the study is relatively
small, this finding indicates that the reform/modeling approach empowers students who traditionally are excluded in mathematics or do not succeed due to one reason or another.
The findings on gender differences in the pre-calculus study are consistent with findings on
middle and high school students performance in mathematics (Fennema et al., 1998; Mullis, et al.,
1998; Beaton, et al., 1996; Hyde, et al., 1990). This means that there is a link between mathematics
performance at the school level and mathematics performance at the college level. Although the
Page 32
pedagogical issues and context are different, students from middle and high schools do go on to college and by understanding the causes of these gender and racial difference in mathematical performance, we can design better instructional programs to minimize or eliminate these differences
wherever they may be.
Page 33
Appendices
Appendix I. Pre-Calculus Study: Ten Common Questions from Final Examination
Appendix II: Results of LICIL Student Attitudinal Survey in Precalculus
Page 34
Page 35
Page 36
Appendix II
New York Institute of Technology
Fall Semester 1999
Report on Pre/Post Surveys in Pre-calculus Class
Leo Gafney
Project Evaluator
Surveys were administered to students in four pre-calculus classes at the New York
Institute of Technology at the beginning and end of the fall, 1999 semester. These surveys
were used as part of a more general evaluation of the NSF-funded grant to the Long Island
Consortium for Interconnected Learning in the Quantitative Disciplines (LICIL).
The survey was an adaptation form of an instrument originally developed by the
evaluators from five of the systemic math consortia. Adaptations were made in the light of
returns from previous years, eliminating items that had not been fruitful and including
others that covered areas more pertinent to LICIL activities.
The survey was intended to explore students’ experiences and attitudes toward
mathematics. The following table shows the four classes as identified by the institution
and the number of students completing the pre and post surveys.
Math 140/WO1 Precalc, reform text
Math 140/WO2 Precalc, traditional text
Math 140/WO3 Precalc, reform text
Math 140/WO4 Precalc, traditional text
Total
Pre
26
9
25
25
85
Post
14
12
19
10
55
The survey included 20 items. In the pre-course survey the items were worded in
general terms, for example, “In most courses that use math we learn by doing not just listening.” In the post-course survey items were worded to pertain specifically to the class in
which the survey was administered, for example, “In this course we learned by doing, not
just listening.” The rationale was that the pre-course data would supply baseline data for
student attitudes and experiences in general. It was hoped that the post course survey
would provide a profile of student experiences for that course, standing out against the
more general attitudes.
When a similar pre/post survey was used several years ago no significant differences were found when comparing pre/post data. But there were differences based on
math level, with students in higher level courses generally indicating more positive attitudes about mathematics.
Page 37
The 20 items on the survey can be clustered in several general areas, and responses
analyzed according to these areas. The first area is whether mathematics is an active,
open-ended, discovery-oriented process or a passive, closed, memory-based procedure.
The following six items belong to this category. The wording below is from the post-test.
Appropriate changes are made to generalize the wording for the pre-test, as discussed
above.
1. In this course we learned by doing, not just listening.
2. This course required me to ask “why.”
7. In this course we mainly memorized facts and rules.
9. This course got me to try new ways of thinking and learning.
12. This course showed me that there can be more than one right answer.
13. This course required a lot of exploration and experimentation.
The survey included a five-level Likert-scale with responses from 1 for ‘disagree
strongly’ to 5 for ‘agree strongly.’ For purposes of analysis disagree (1 and 2) were collapsed, as were agree responses (4 and 5). Responses for item 7, above, were reversed in
calculating averages; that is disagree is considered positive and so percent disagreeing
was averaged with percents agreeing for the other items. The table below shows the average positive response for the six items listed and the change for each class from pre to post
survey.
Percent Positive Responses to Math as Open, Active, Discovery-Based
Math 140/WO1 Precalc, reform text
Math 140/WO2 Precalc, traditional text
Math 140/WO3 Precalc, reform text
Math 140/WO4 Precalc, traditional text
Average
Pre
60.5
57.3
58
56
57.95
Post
63.2
47.2
66.7
41.7
54.7
Change
+ 2.7
- 10.1
+ 8.7
- 14.3
- 3.25
The pre-course responses are clustered close to the mean, with a range of only 4.5
and a standard deviation of 1.9. In other words, regarding the scale of these six items, the
four groups are comparable. The post-course responses, however, are not clustered, with a
range of 25 and a standard deviation of 12.1. Group 1 showed a modest increase, group 3
a substantial increase in the direction of more positive attitudes and experiences. Groups 2
and 4 showed substantial movement in a negative direction regarding attitudes and experiences about mathematics as an open, active, discovery-based process. Average pre/post
means for the two reform/modeling text classes moved from 59.3 to 65; while average
pre/post means for the traditional text classes fell from 56.7 to 44.5.
Page 38
A second general area contains four survey items that ask about the utility of mathematics and its connectedness in situations beyond math courses. The items in this area
are as follows.
3. This course helped me to understand how to apply math to real world problems.
5. In this course, I learned ways of thinking that are useful in situations outside of
math.
8. This course showed that math is useful in many non-math courses.
17. This course made connections across disciplines.
The table below shows the average positive response for the four items listed and the
change for each class from pre to post survey.
Percent Positive Responses to Math as Useful and Connected
Math 140/WO1 Precalc, reform text
Math 140/WO2 Precalc, traditional text
Math 140/WO3 Precalc, reform text
Math 140/WO4 Precalc, traditional text
Average
Pre
53.3
65.5
57.5
52
57.1
Post
38
6
68.3
12.5
31.2
Change
-15.3
-59.5
+10.8
-39.5
-25.9
Percent of positive responses for the pre-course survey are again clustered, although not as closely as in the previous area, with a range 13.5 and a standard deviation of
6.1. Post-course results are scattered even more widely than in the first area, with a range
of 62.3 and a standard deviation of 28.3. Once again, groups 1 and 3 diverge from groups 2
and 4 in pre/post scores. Average pre/post means for the two reform/modeling text classes moved from 55.4 to 53.2; while average pre/post means for the traditional text classes
fell from 58.75 to 9.25.
Based on the responses to the two general areas summarized above it appears that
students respond much more positively to the reform/modeling-text courses than to the
traditional-text courses. These findings are useful because the reform/modeling calculus
and pre-calculus texts and methods emphasize these areas of active learning and applied/connected math. One might hypothesize that differences in the student populations
or in the instructors accounted for the differences. Based on responses to other items we
can, to some extent, reject these hypotheses. We will summarize these findings next.
Items 4 and 18 can be considered in the general area of confidence in doing
math.
4. Because of this course, I have confidence in my ability to do well in other math
courses.
18. When I get stuck on a math problem I can usually finally solve it.
Page 39
Positive responses to these items are shown in the following table.
Student Confidence in Doing Mathematics
Math 140/WO1 Precalc, reform text
Math 140/WO2 Precalc, traditional text
Math 140/WO3 Precalc, reform text
Math 140/WO4 Precalc, traditional text
Average
Pre
67
38.5
45
59.5
52.5
Post
29
46
60.5
35
42.6
Change
-38
+ 6.5
+15.5
-24.5
-10.1
This area shows a rather wide variation both from pre to post course and from class
to class, without regard for the use of reform/modeling or traditional text. The fact that
there are substantial changes from pre to post surveys would seem to indicate that the individual class had an effect. With the available data it is not possible to say whether the
changes should be attributed to the instructor or other factors associated with the course.
One item on the survey pertained explicitly to technology. On the pre-course survey, this was worded, “The computer and graphing calculator are very helpful for learning.” In the post-course survey it read.
15. Because of this course, I have a better understanding of how important
the computer and/or graphing calculator are in learning.
Positive responses are shown in the following table.
Importance of the Computer and Graphing Calculator
Math 140/WO1 Precalc, reform text
Math 140/WO2 Precalc, traditional text
Math 140/WO3 Precalc, reform text
Math 140/WO4 Precalc, traditional text
Average
Pre
88
78
76
87
82.3
Post
79
67
100
70
79
Change
-9
-11
+24
-14
-3.3
It is interesting to note that the average for group 3 was the lowest in the pre-course
survey but increased to 100 percent positive rating, while the other three groups all declined in percent of positive response.
One item on the survey was about the concern of teachers. In the pre-course survey
it was worded, “At this college the teachers of math and science really care about whether
or not you learn.” In the post-course survey it was worded as follows.
19. In this course the professor cared about whether or not you learned.
Page 40
Results are shown in the following table.
Concern of Teachers About Learning
Math 140/WO1 Precalc, reform text
Math 140/WO2 Precalc, traditional text
Math 140/WO3 Precalc, reform text
Math 140/WO4 Precalc, traditional text
Average
Pre
50
50
76
33
52.3
Post
54
75
100
60
72.3
Change
+4
+25
+24
+27
+20
It is encouraging to note that although there is a wide range in the post-course responses, positive attitudes increased for all classes. These results do not correlate with
membership in the reform/modeling-text and traditional-text courses. This seems to indicate that students in fact consider the items about mathematics on their own terms, and do
not respond positively about understanding or the usefulness of math simply because
they like the instructor.
Summary of Findings and Conclusions
There are a number of significant differences between the results of the pre and
post surveys and from class to class. The following can be stated based on the data and
analysis above.
*
Those in the reform/modeling-text classes were significantly more positive in their
attitudes about mathematics as an active, discovery-based experience than those in the
traditional-text classes.
*
Those in the reform/modeling classes were significantly more positive in their attitudes about mathematics as useful and connected to other disciplines than those in the
traditional-text classes.
*
In the areas of confidence doing math and in appreciating the importance of technology, group 3 increased significantly more than the other groups and had significantly
higher final post-course results than the other groups. Based only on the survey it is difficult to assign reasons for these outcomes. It seems, however, reasonable to infer that the
instructor for group 3 was particularly supportive with regard to student learning and
used the computer and/or graphing calculator effectively. It seems fair also to conjecture
that the reform/modeling-text instruction works better in the hands of some instructors
than others.
Page 41
References
Beaton, A. E., Martin, M. O., Mullis, I. V. S., Gonzalez, E. J., Smith, T. A. and Kelly, D. L.
(1996). Science Achievement in the Middle Years: IEA’s Third International Mathematics and Science Study. Chestnut Hill, MA: TIMSS International Study Center,
Boston College.
Fennema, E, Carpenter, T., Jacobs, V., Franke, M. L., and Levi, L. W. (1998). A Longitudinal Study of Gender Differences in Young Children’s Mathematical Thinking. Educational Researcher, Volume 27, Number 5, pp. 6-11.
Gordon, F. (1995) Assessing How Well PreCalculus Students Do While Functioning in the Real
World, PRIMUS.
Gordon, F. (2000) Personal communication on MathFest presentation, April 28, 2000.
Hamer, R., Wielenga, D., and Jayawickrama, J. (1998). Multivariate Statistical Methods Using the JMP Software. Cary, NC: SAS Institute Inc.
Hyde, J. S., Fennema, E., and Lamon, S. J. (1990). Gender Differences in Mathematics Performance: A Meta-Analysis. Psychological Bulletin, Volume 107, Number 2, pp. 139155.
Long Island Consortium. (1997). Consortium Gathers Steam: Update from Consortium Director. Long Island Consortium for Interconnected Learning Newsletter, Volume 3, No.
3, pp. 1-3.
Mullis, I. V. S., Martin, M. O., Beaton, A. E., Gonzalez, E. J., Kelly, D. L. and Smith, T. A.
(1998). Mathematics and Science Achievement in the Final Years of Secondary
School: IEA’s Third International Mathematics and Science Study. Chestnut Hill,
MA: TIMSS International Study Center, Boston College.
SAS Institute (1995). JMP Statistical Discovery Software. SAS Campus Drive, Cary, NC:
SAS Institute Inc.
Tukey, J. W. (1977). Exploratory Data Analysis. Reading, MA: Addison-Wesley.
Page 42
Kabba:
I’ve gone through your report completely and, as you suggested, I’ve taken the liberty of adding a number of piec
that I mentioned in my earlier note to you and that I feel are essential. I’ve also clarified a number of item
that I think require a little more background information. I’ve also made a number of corrections. All
these changes appear in blue so you can find them easily in the attached document. Obviously, though, y
will have to change the color back to black.
In particular:
1. I added a series of bulleted items in the introduction on Summary of findings. All of these items appear in v
ious scattered places throughout the report, but I think it is essential to summarize all these aspects together
front. Most of my colleagues will not read through the full report, unless something captures their attention
the start.
2. I’ve incorporated the three questions out of the 10 common questions on the final exams in precalculus in F
1999 where there was a significant difference. I also added a discussion on some of the statistical measures
lated to each question.
3. I’ve added in the entire list of student responses to the question asking for the meaning of the slope from t
problem on the final exams as a new table, Table 3A and 3B. I feel that the nature of the student responses
the practical meaning of the slope of a line is an unbelievably important item because it goes to the heart
what they understand. If they can’t interpret the slope in this kind of contextual problem, they cannot use t
mathematics in any other course, in any other discipline, intelligently. This one item has tremendous implic
tions across all of mathematics education – it strongly indicates that we cannot presume that merely teachi
the mechanics of any procedure is enough and that the students will be able to make the connections betwe
the technique and the application on their own. Perhaps even more should be said about this issue in your
port. It should certainly be emphasized in any articles based on the study.
4. I found that the figures you used to compare the average scores on the 7 quizzes (Figure 5) and the avera
scores on the 3 class tests (Figure 8) were very ineffective, especially when printed in black and white, so th
the colors are not there. I’ve taken the liberty of replacing these two figures with new figures that show t
trends over the entire semester.
5. The issue of students missing quizzes focused on the wrong issue in your discussion. You focused on the s
tistical aspects of missing data – that because scores were not there, you could not draw too many quantitati
conclusions. It is not so much an issue that the scores were “missing”, but rather that the students gave up a
did not take the quizzes, or completely dropped the course. The real issue here is student persistence and r
tention, two major concerns of the institution and the mathematics community in general. As a rule, natio
wide, the success rate in the traditional precalculus courses is probably well under 50% -- there are huge dro
out problems across the country. The fact that so many students gave up and stopped attending the tradition
classes is typical. The fact that so few gave up and stopped attending the reform/modeling courses is a d
matic and important achievement. So I’ve changed the discussion to focus on student persistence and rete
tion and have added a figure (Figure 7a and 7b) to demonstrate this information much more dramatically. P
haps even more should be added to the report on this issue.
Page 43
6. The previous study that I did took place about six years ago. I referred to it in the discussion and added a ref
ence to a paper I sent you some time ago. In this regard, several of your references were not in alphabetical o
der, so I re-ordered them.
As for specific changes:
1. I changed Long Island Consortium to LICIL throughout. This is how the project is referred to.
2. I changed “reform” to “reform/modeling” throughout since the focus in the reform course is mathematic
modeling and the word “reform” sometimes has funny connotations and people react badly at the suggesti
that they have been doing something wrong.
3. I occasionally moved paragraphs around (without highlighting in color) to keep the items in a more chronolo
cal order. I feel that it makes more sense to read in this order. For instance, the discussion on placement te
should precede, since the students take the placement test before they even start the course.
I also amplified some of the discussions to add extra background and/or explanations. For instance:
1. In Spring, 2000 calculus, why there were only 13 students who had come from the reform/modeling bac
ground. Not all students are required to go on to calculus and even those who do will not necessarily do so t
following semester.
2. That placement tests take place before the start of the semester to place the students into what we hope will
the correct course based on their mathematical background.
3. The “missing” quizzes again.
What else needs work:
1. Check the placement of all figures. Figures 5 and 6 appear too early, well before the associated discussion
Figure 8 appears much too late. Etc.
2. Some of the paging is messed up because of the changes. Please correct.
3. Several of the original figures seem to have gotten messed up with what appears to be double imaging in th
text. I don’t know what I may have done to create this and could not correct it.
4. I have mailed you a two page typed copy of the 10 common questions, numbered 1 through 10 to correspon
to what you have in the report. Please remove the entire test in the appendix (it is not necessary). Also, re
move your reference to question #1, #3, #5, #6ii, etc on page 45.
5. In Figure 5, you refer to “standard deviations in parentheses”, but none are shown. Either remove the refe
ence or add the sd’s.
6. In your figure 3, there is something about “bkg”. I don’t know what this refers to. Can you either amplify
change the heading.
7. It is also necessary for you to change the list of figures and tables – I didn’t touch that file.
8. On the front page, please change Long Island Consortium to “Long Island Consortium for Interconnecte
Learning”.
Finally, I’ve been in touch with Bill McGuinness several times and he assures me that you will be paid muc
more promptly than the last time. He says they have gotten the bugs out of their system.
I look forward to seeing the final report soon. Unless I’m wrong, it seems very close to completion. (Please sen
me a copy before you finalize it so I can make one more pass through it.) If you have any questions or want t
discuss any of these issues with me, please don’t hesitate.
Page 44
I’d like to distribute the final report as soon as possible – well before the end of the semester when people hav
finals and the like on their minds, if possible.
Thanks again for all of your work and help on this study.
Florence
Page 45