Effects of Curricular Change in a
Freshmen College Applied
Algebra Course
Dr. Robert Mayes
Director of the
Institute for Mathematics Learning
Crisis in Mathematics Education


Business Higher Education Forum (2005)
American Diploma Project (2005)





Percentage of ninth grade students in the U.S. who
graduate from high school is 68%
40% of these same ninth graders start college
27% of them persist through the second year
18% earn a degree
22% of college freshmen are not ready for the entry
level mathematics course and require remediation
Crisis in Mathematics Education

MAA Task Force on the First College Level
Mathematics Course (Kime et al, 2000)




College enrollments are increasing, calculus
enrollments are stagnant
9% of students matriculate into calculus
Majority of first year students’ first college
mathematics course is either remedial, liberal arts, or
college algebra
Failure and withdrawal rates in before calculus
courses are often dismal, with numbers between 40%
and 60% common
Institute for Mathematics Learning

Reform in before calculus courses over
the past 5 years


Liberal Arts Mathematics, Applied College
Algebra, College Algebra, College
Trigonometry, Precalculus, and Applied
Calculus
Subsequent Course Success

Fall 2004-Spring 2005 students successful in a
subsequent course 80% if A or B in IML
course, 50% if C in IML course
Success Rates
Fall 2004-Spring 2005
Course
Liberal Arts Mathematics
Success Rates
(A, B, or C grade)
65.9%
Applied Algebra
63.3%
College Algebra
58.6%
College Trigonometry
63%
Precalculus
64%
Applied Calculus
68%
Applied College Algebra Study






Theoretical Framework
Curricular Revision
Method
Pilot Study – Fall 04
Full Study – Spring 05
Discussion
Theoretical Framework





Constructivist theory of learning
Curriculum and Evaluation Standards
(2001)
Achieving Quantitative Literacy (2001)
CUPM Curriculum Guide for
Undergraduate Courses in Mathematical
Sciences (2003)
A Collective Vision: Voice of Partner
Disciplines (2004)
Theoretical Framework



Tapping America’s Potential: The
Education for Innovation Initiative
(Business Round Table, 2005)
A Commitment to America’s Future:
Responding to the Crisis in Mathematics
and Science Education (Business Higher
Education Forum, 2005)
American Diploma Project (Achieve, Inc.,
2005)
Theoretical Framework

Liberal Studies Program goals




Introduce the great ideas and controversies in
human thought and experience, in this case
the function concept.
Develop the ability to reason clearly,
communicate effectively, and understand
major influences of society.
Develop critical thinking by requiring logical
inquiry to evaluate decisions, question posing,
problem formulation, and interpretation of
results.
Incorporate a writing component
Theoretical Framework

Cognitive science



Lyn English (1997) views reasoning as embodied and
imaginative
Learner Centered Instruction movement
Treisman’s Model (1992)




Collaborate on challenging problems in an
environment of high expectations
Weekly collaborative learning sessions in small groups
with student mentors
Rather than remediate - engage in challenging
mathematics that is engaging and meaningful
Faculty sponsorship in development and management
of the courses
Curricular Revision

Computer enhanced


Vista WebCT course management software,
web sites, and interactive Java applets to
provide access to course materials, implement
assessment, communicate with students,
engage students in exploring and discovering
mathematics, and manage the course grades
Grapher
Curricular Revision


10 online quizzes, four on-line chapter
reviews, four on-line exams, and an online gateway pre-assessment to determine
student mathematical deficiencies
Computer laboratory component

Students are peer mentored while they
engage in explorations of mathematical
concepts and apply mathematics to solve real
world problems
Curricular Revision


Active student learning and student
accountability, implementing teaching strategies
that engage students and provide informal
formative feedback on their progress
Personal Response System (PRS)


Implement 22 classroom participation activities that
allow students to respond and receive immediate
feedback
Power point slides guide the course discussion, serve
as student lecture outlines, and provide instructors
with real world data problems as well as a guide to
key concepts
Curricular Revision

Supplemental Instruction (SI)



Fall 2004 - paper worksheets focusing on
selected skills or applications
Spring 2005 grounded in programmed
instruction (McHale, Christenson, and Roberts,
1986) and implemented with PRS
Two versions of SI


Algorithm SI focus on basic skills without
context
Application SI used real world applications to
motivate the need for learning basic skills
Curricular Revision

Students assigned to SI based upon performance on
Pre-assessment.




Taken after a week of reviewing college algebra prerequisites
Students who scored below 80% on the first attempt were
required to attend an SI review session and then retake the Preassessment within a week.
If the student failed to attain an 80% or higher mark on the Preassessment on either attempt, then they were required to attend
SI for the remainder of the semester (designated Required)
Students who scored above 80% were encouraged to attend SI,
but their attendance was not required (designated Optional)
Optional students whose performance on one of three
subsequent tests was less than 70% were then required to
attend SI until they earned at least 70% on a later test
Method




Class randomly assigned to receive either
Algorithm SI or Application SI
Retired version of the standardized ACT
Mathematics exam as Pre- Post-test
Mathematics Attitude Inventory (MAI),
developed by Mayes (2004)
Analysis of impact of supplemental on student
outcomes within course components, including
exams, laboratories, on-line homework quizzes,
and overall grade.
Method







Nonequivalent Control Group Quasi-experimental
research design
O1 O2 O3 X1 O4 O5 O6
--------------------------------------O1 O2 O3 X2 O4 O5 O6
O1 - O4 : Pre- and Post-Math Attitude Inventory (MAI)
O2 – O5 : Pre- and Post-ACT Exam
O3 : Pre-assessment of basic skills
O6 : Final Exam
X1 : Algorithm SI
X2 : Application SI
Method

Analysis was conducted using quantitative
methods, including univariate and
repeated-measures analyses of variance
(ANOVA) and correlations. Significant main
effects were further analyzed with
pairwise comparisons using a Bonferroni
adjustment.
Research Questions



What are the plausible effects of a
Supplemental Instruction program
targeted at students at risk of failure?
Does Application SI or Algorithm SI have
the greatest impact on student cognition
and affect?
What is the impact of the reformed
Applied College Algebra course on
student cognition and affect?
Pilot Study Fall 04


ACT Pre- Post Analysis
2 x 2 repeated-measures ANOVA with a
between-subject factor of SI Requirement
(Required or Optional) and a within-subject
factor of Test (ACT pretest and ACT posttest)


No significant difference between the cohorts on
overall ACT score
Significant main effect of ACT, F (1, 283) = 335.695,
p < .001, with the posttest scaled scores (M = 19.86,
SD = 3.02) exceeding the pretest scaled scores (M =
15.71, SD = 3.11)
Pilot Study – ACT Analysis
Cohort
N
Pre-test
Mean
Pre-test Std.
Dev.
Post-test
Mean
Post-test Std.
Dev.
Optional
Earned 1-5
9
13.00
3.39
17.89
3.48
Optional
Earned 6-9
38
15.18
2.95
19.34
2.76
Optional
Earned 10-11
123
15.80
3.04
19.98
2.95
Required
Earned 0
17
15.41
3.20
19.06
3.67
Required
Earned 1-5
30
16.10
2.81
19.80
3.22
Required
Earned 6-9
36
16.44
3.45
20.14
3.32
Required
Earned 10-11
37
15.76
3.08
20.57
2.44
Pilot Study – ACT Subscales

Subscales of Pre-Algebra and Elementary
Algebra (PAEA) and Intermediate Algebra
and Coordinate Geometry (IACG)


All Optional cohorts made significant gains on
both subscales
Required cohorts made significant gains in
both subscales when they participated in SI
over 50% of the time
Pilot Study – ACT Subscales
Cohort
Mean Gain in Std. Dev. In
PAEA
PAEA
Mean Gain in Std. Dev in
IACG
IACG
Optional
Earned 1-5
1.857*
2.93
2.143**
2.179
Optional
Earned 6-9
2.478***
3.082
2.957***
2.675
Optional
Earned 10-11
2.415***
2.985
2.468***
2.659
Required
Earned 0
-0.333
4.163
3.667
3.055
Required
Earned 1-5
2.200*
2.300
1.600
2.875
Required
Earned 6-9
2.969***
3.290
2.031***
2.456
Required
Earned 10-11
2.774***
3.074
2.355***
2.374
Pilot Study - Final Course Average





Univariate ANOVA indicated a significant effect of Cohort on final
course average, F (6, 398) = 6.970, p < .001.
Pairwise comparisons using a Bonferroni adjustment indicated that
both the Required and Optional 6-9 cohorts outperformed the
Required 0 cohort, p <.05.
Required and Optional 10-11 cohorts outperformed the Required 0
cohort, p < .001.
No significant differences were found between Required and
Optional students who earned the same number of points for SI.
There is also overwhelming evidence that the more you attend SI,
the better you do in the course.


A Required student attending SI 6 to 9 times has a course mean
equivalent to an Optional-Earned 6-9 student
A Required student attending 10 to 11 times actually had a higher mean
than an Optional-Earned 10-11 student.
Pilot Study - Affect

Statistically significant drop in overall
students’ attitudes from the beginning to
end of the semester


Paired-samples t-test (t (459) = 11.92, p<
.001).
Required SI students had a significantly
poorer attitude at the end of the semester
then their Optional SI counterparts
Pilot Study - Affect
Optional
Mean
Attitude
Survey 1
Attitude
Survey 2*
Required
Std.
Deviation
Mean
Std.
Deviation
3.03
.81
2.91
.93
2.32
1.40
2.00
1.43
Full Study – Spring 2005

PRS in Supplemental Instruction and
offering Algorithm SI and Application SI at
distinct times in large lecture classrooms,
allowed more control over tracking
students. The question of which type of
SI was most effective could now be
addressed.
Full Study – Spring 2005


A 2 x 2 x 2 ANOVA with between-subjects
factors of Section (Application or
Algorithm) and SI Requirement, and a
within-subjects factor of Test was used to
analyze the data.
There were no significant differences
between the Application SI and Algorithm
SI sections on the ACT pretest or posttest
Full Study – Spring 2005



Significant main effect of Test, F (1, 205)
= 180.471, p < .001, such that
performance on the posttest (M = 20.15,
SD = 2.47) was better than on the pretest
(M = 18.42, SD = 2.64)
Significant main effect for SI Requirement,
F (1, 205) = 11.973, p = .001, with the
Optional group (M = 19.875)
outperforming the Required group (M =
18.89).
There were no significant interactions.
Mean scaled ACT scores and standard
deviations by section and Supplemental
Instruction requirement
Section
Supplemental
Instruction
Requirement
ACT Pre
Mean
ACT Pre
SD
ACT Post
Mean
ACT Post
SD
Algorithm
Optional
19.44
2.91
21.93
2.35
Required
18.09
2.34
19.26
2.10
Optional
18.61
2.49
20.37
2.45
Required
18.14
2.92
20.16
2.51
Application
Three Subscales of ACT
overall effect



PAEA subscale: post-test (M = 16.82, SD=3.21)
exceeding pre-test performance (M = 14.66,
SD = 3.67), F(1, 205) = 85.067, p < .001
IACG subscale: post-test (M = 8.44, SD = 2.69)
exceeding pre-test performance (M = 6.63, SD
= 2.64), F(1, 205) = 97.903, p < .001
PGTRG subscale: post-test (M =7.54, SD=2.83)
exceeding pre-test performance (M = 6.29, SD
= 2.64), F(1, 205) = 43.40, p < .001
Three Subscales of ACT
by SI Type



PAEA subscale: Optional SI (M = 17.615)
outperformed Required SI (M = 15.12),
F(1, 205) = 28.974, p < .001
IACG subscale: Optional SI (M = 8.69)
outperformed Required SI (M = 7.155),
F(1, 205) = 18.442, p < .001.
PGTRG subscale: Optional SI (M = 8.02)
outperformed Required SI (M = 6.545),
F(1, 205) = 16.319, p < .001
ACT Subscale by SI Type
Subscale Section
SI
ACT
requirement Pre Mean
ACT
Pre SD
ACT
Post Mean
ACT
Post SD
PAEA
Optional
16.35
3.35
18.42
3.10
Required
14.21
3.49
16.36
3.26
Optional
16.50
3.52
19.19
2.12
Required
13.90
3.69
15.93
2.92
Optional
7.13
2.60
9.88
2.42
Required
6.32
2.42
7.98
2.91
Optional
7.50
3.25
10.08
2.33
Required
6.44
2.60
7.87
2.19
Optional
7.77
2.76
8.27
2.43
Required
6.07
2.64
7.76
2.86
Optional
6.88
2.72
9.15
2.84
Required
5.79
2.36
6.39
2.50
Algorithm
(24)
Application
IACG
Algorithm
(18)
Application
PGTRG
Algorithm
(18)
Application
ACT by Cohort



Optional 0 students outperformed the
Optional 1-5, Optional 11-13, and all
Required cohorts on the ACT.
No other cohorts were significantly
different from one another.
This supports the previous findings that
while the overall course influenced ACT
posttest scores, SI had little impact
ACT by SI Attendance
Cohort
N
Mean ACT
SD
Optional 0
27
21.204
.429
Optional 1-5
30
19.017
.407
Optional 6-10
17
19.882
.541
Optional 11-13
10
18.850
.706
Required 0
28
19.268
.422
Required 1-5
37
19.338
.367
Required 6-10
28
19.036
.422
Required 11-13
32
17.922
.394
Final Course Average by Cohort




All Optional cohorts outperformed both
the Required 0 and Required 1-5 cohorts
Required 0 cohort was outperformed by all
other Required cohorts
No other cohorts were significantly
different from one another
This indicates that SI had an overall
course impact for Required students who
attended at least 50% of the time
Final Course Average by Cohort
Cohort
Optional 0
N
34
Mean Course Avg
75.032
SD
22.507
Optional 1-5
39
73.509
16.575
Optional 6-10
Optional 11-13
Required 0
18
12
67
79.099
80.728
36.764
9.394
8.362
31.540
Required 1-5
63
53.094
26.436
Required 6-10
46
59.960
23.722
Required 11-13
37
66.403
14.392
Exam Average by Cohort




Optional 0 cohort outperformed all of the
Required cohorts
Optional 1-5 cohort outperformed the
Required 0 and Required 1-5 cohorts
Optional 6-10 and Optional 11-13 cohorts
outperformed the Required 0, Required 15, and Required 11-13 cohorts
Required 6-10 cohort significantly
outperformed the Required 0 cohort. No
other comparisons were significant.
Exam Average by Cohort
Cohort
N
Mean Exam Avg
SD
Optional 0
34
73.11
22.63
Optional 1-5
37
64.48
16.62
Optional 6-10
17
76.45
9.45
Optional 11-13
12
74.41
13.48
Required 0
62
40.46
27.64
Required 1-5
61
49.58
25.89
Required 6-10
42
56.56
20.68
Required 11-13
37
49.70
18.91
Lab Average by Cohort



All Optional cohorts outperformed the
Required 0 cohort
Required 6-10 and Required 11-13 cohorts
outperformed the Required 0 cohort
No other comparisons were significant
Lab Average by Cohort
Cohort
N
Mean Lab Avg
SD
Optional 0
34
77.97
26.18
Optional 1-5
39
78.48
27.77
Optional 6-10
18
87.73
22.62
Optional 11-13
12
86.02
12.08
Required 0
67
51.30
39.26
Required 1-5
63
65.50
32.33
Required 6-10
46
71.75
30.18
Required 11-13
37
78.46
26.03
Exam by Exam

In these analyses the cohorts are
determined by the most previous test
performance


Exam 1 (2 SI sessions) Optional 0 cohort
outperformed all Required cohorts
Exam 2 (4 SI sessions) all Optional cohorts
outperformed the Required 1 and 2 cohorts.
The Optional 0, 1, and 3 outperformed the
Required 3 and 4 cohorts. Only the Optional 0
cohort significantly outperformed the
Required 0 cohort. Required 0 cohort
outperformed the Required 2 cohort
Exam by Exam

In these analyses the cohorts are
determined by the most previous test
performance


Exam 3 (4 SI sessions) Optional 0, 1, 2, and 3
cohorts outperformed all Required cohorts on
Exam 3. Optional 4 cohort significantly
outperformed Required 0 and 1 cohorts
Final Exam (3 SI sessions) Optional 0 and 3
cohorts outperformed all Required cohorts.
Optional 1 and 2 cohorts outperformed
Required 0, 1, and 3 cohorts.
Exam by Exam

Overall the examination of SI exam by
exam reveals that Optional SI students
continue to outperform their Required SI
counterpoints. Surprisingly, Required SI
students who attend all of the required
sessions did not score significantly better
on the following exam, although the trend
was that exam scores did increase as
attendance increased.
Attitude Subscales





A paired-samples t-test was conducted on the
average responses for the Attitude Survey
No significant difference between the two
administrations of the Attitude Survey
No significant correlation between students’ final
grades and their responses on the Attitude
Surveys
There were some general effects on the Utility,
Locus of Control, and Belief subscales, where
the pretest scores were more positive than the
posttest scores
There were no significant differences between
the Application SI and Algorithm SI sections
Attitude Subscales
Attitude Survey 1
Attitude Survey 2
Subscale
Number of
Questions
Mean
SD
Mean
SD
*Utility
7
24.73
4.694
23.19
5.389
Concept vs.
Skill
10
29.83
4.95
29.62
4.94
*Locus of
Control
6
19.50
4.507
18.25
4.704
*Belief about
Math
3
6.12
2.611
5.78
2.681
Technology
4
15.03
2.866
15.06
3.581
Discussion