Ready, Set, Register: Using Course Over-Enrollment to Examine the Causal Impact of
Mathematics Course on College Outcomes
by
Jenna Cullinane
March 1, 2013
Abstract
In response to insufficient degree completion and inequitable distribution of degrees,
policymakers, philanthropists, and educators are calling on college and universities to
dramatically improve their graduation rates. These education stakeholders seek levers to
improve postsecondary student achievement through policy. Mathematics preparation and
course-taking in college are key indicators of postsecondary degree completion; however,
weaknesses in prior studies of this relationship raise concerns about the appropriateness of
policies that encourage math course-taking as a means of improving college success. The fact
that high achieving students are more likely to take mathematics early and often in college bias
previous results. This paper examines how the timing and completion of math course-taking in
college affects degree completion using regression discontinuity in course registration data.
Surprisingly, descriptive statistics from a single large, public institution in Texas suggest
students who take math in their first semester of college have a lower likelihood of graduation.
This phenomenon may be explained by high performing students satisfying college
requirements in high school through growing AP credit and dual credit opportunities in Texas.
Among students who take math in their first semester, the likelihood of completion is positively
correlated with the level of first math course taken. Students who begin their college career
with more advanced mathematics are more likely to graduate. Quasi-experimental findings are
forthcoming.
Cullinane, College Math—1
Ready, Set, Register: Using Course Over-Enrollment to Examine the Causal Impact of
Mathematics Courses on College Outcomes
More than 75 percent of new jobs in the U.S. will require workers with a postsecondary
credential, and 60 percent of the fastest growing jobs in the next five years will require at least
a postsecondary certificate (The Whitehouse, 2009; BLS, 2007). Today, just 39 percent of
Americans have earned any type of postsecondary credential. Degree completion is unequally
distributed across racial, ethnic, and socio-economic subpopulations (U.S. Census Bureau, 2010;
Dowd, 2003). In response to insufficient degree completion and inequitable distribution of
degrees, policymakers, philanthropists, and educators are calling on college and universities to
dramatically improve their graduation rates. In particular, education stakeholders seek levers
to improve postsecondary student achievement through state and institution-level policy.
Recent studies point to the sequence and timing of college courses and their alignment
to programs of study as important factors in improving degree completion. They find that
success in mathematics courses early and often in high school and college is predictive of later
educational outcomes (Long et al., 2009; Xin Ma & Wilkins, 2007; Adelman, 2006). For
example, advanced mathematics course-taking beyond Algebra 2 in high school and early math
course-taking in college are associated with completing a bachelor’s degree (Adelman, 2005;
Long et al., 2008). The completion of a first college-level course is also highly correlated with
the probability of earning a two-year degree (Roska & Calcagno, 2008).
The frequency and consistency of findings linking mathematics to later educational
success has encouraged policymakers to mandate or otherwise incentivize colleges and schools
Cullinane, College Math—2
to provide more mathematics instruction, more advanced content, to more students in hopes
of improving attainment. The Algebra for All movement in K-12 education is one such policy
that demonstrates the consequences of policymaking in the absence of sufficient causal
research. Algebra for All accelerated entry into algebra courses for all students in the 8 th grade
based on positive associations between Algebra and later educational outcomes for generally
high-achieving students. The policy presumed the association between mathematics and later
outcomes was causal, that is, mathematics is a mechanism that improves outcomes, and by
increasing mathematics outcomes will increase as a result. Unfortunately, the studies upon
which these policies are based have not addressed the problem of nonrandom selection of high
ability or better-prepared students into mathematics courses in high school and college, nor did
they account for student major in analyses of course-taking patterns and Algebra for All has
largely been deemed a failure.
At best, existing literature concludes that students that have completed degrees or
enrolled in college also tend to be more successful in mathematics. When the standard for
causal claims has not been met, researchers and policymakers must be cautious to accurately
describe and interpret findings on the basis of associational research. Policymakers are already
crafting performance funding policies and incentive funding policies for higher education that
reward college math completion and the lessons of Algebra for All should not be forgotten.
I attempt to address limitations in the mathematics educational literature and inform
policies designed to improve college graduation by conducting a rigorous quasi-experimental
study that identifies how first college-level mathematics courses and their timing affects the
probability of graduation. A unique research design using as-if randomization of student
Cullinane, College Math—3
mathematics course-taking at Texas university permits causal inference and addresses selection
problems that previous studies have struggled to overcome. Excess demand for courses and
new data on student efforts to register for courses provide a unique opportunity to compare
students who enroll in a college level mathematics course with those who were not able to
enroll in the first semester due to the class size limitations using a regression discontinuity
research design (Thistlethwaite and Campbell, 1960). These students have the same revealed
preference for taking a particular course—that is, they all attempted to register for a math
course in the first semester—and we can assume successful enrollment among this population
is random. The present study will further address shortcomings of the existing literature by
examining effects by major and within a single institutional context.
I hypothesize that early math course-taking has only a modest impact on graduation, as I
suspect selection effects have inflated previous findings. Further I expect the effect of early
math course-taking will vary by major.
Literature Review
A review of the college persistence and degree completion literature indicates that
mathematics course-taking in high school and college are among the most important, stable
indicators of later success. I began my literature review with searches on JSTOR, Google
Scholar, and the Education Resources Information Center (ERIC) using key words degree
completion, mathematics, predictors, courses and pathways. Initial articles led me to
subsequent resources.
Cullinane, College Math—4
Theory of Student Achievement in College
Human capital theory (Becker, 1975) establishes that education outcomes depend on
perceived future labor market returns to schooling investments. Education and training
increase student ability, therefore we expect that the length, type and quality of schooling to
influence student skills, postsecondary outcomes, and subsequent productivity. Previous
research consistent with the human capital approach suggests mathematics courses build
academic skills and that more highly skilled students will have better outcomes in
postsecondary education. Lessons from the human capital literature at the postsecondary level
are well summarized in the Institute for Higher Education Leadership and Policy’s study,
Student Progress Toward Degree Completion (2009). The authors explain the significant,
positive relationships between mathematics and the likelihood of degree completion (Adelman,
2006; Cabrera, Burkum, La Nasa, 2005), retention (Herzog, 2005), and transfer (Roksa &
Calcagno, 2008; Cabrera, Burkum, & La Nasa, 2005; Adelman, 2005) as the result of developing
key academic skills necessary for subsequent coursework.
Human capital theory explanations are also consistent with findings that taking more
math credits and more advanced math in high school is associated with higher enrollment and
college grades (Schneider, Swanson, and Riegle-Crumb, 1998; Deming, Hastings, Kane & Staiger,
2011) and choice of major (Federman, 2007). There is ample evidence that patterns of math
course-taking, credit accumulation patterns, and choice of major differ for subpopulations of
students such as women and minorities, as do the returns to math courses that explain variable
outcomes across populations (Adelman, 2005; Bailey, Jeong, and Cho, 2008; Spade, Columba,
and VanFossen, 1997; Jacobs, 1996; Kahn & Nauta, 2001).
Cullinane, College Math—5
Economic theory provides a second explanation of the relationship between math and
later college outcomes. Math course taking may serve as a signal, or screening device for
unobserved student ability that admissions officers or faculty use to determine the quality in
applicants to colleges and majors (Benard, 2001). Students with higher achievement tend to
take mathematics courses earlier than students with low achievement. Seventy-one percent of
students who ultimately earn a bachelor’s degree take at least one college-level math course
within the first two years of college enrollment, while just 38 percent of students who not earn
a degree do so (Adelman, 2005). This finding may suggest that mathematics course completion
differentiates high-ability from low-ability students, not that mathematical learning somehow
causes an increase in college ability (Stiglitz, 1975). The previous literature has struggled to
separate the effects of human capital and signaling, and in fact, both theories may be playing a
role. For instance, the fact that recent high school graduates that complete a first college level
mathematics course are four times more likely to complete a two-year degree and older
students who do so are two times more likely to complete a two-year degree (Calcagno et al.,
2007) could be explained either by signaling theory or human capital theory and remains
untested empirically.
Causality and Implications for Policy
While there have been many studies concerning the role of mathematics course-taking
in developing student skills and signaling quality, these studies are uniformly non-experimental
and do not offer opportunities for unbiased causal inference by design or by method. Causality
has three basic requirements: the cause must be related to the outcome, no other alternatives
Cullinane, College Math—6
can explain the effect, and the cause must precede the effect (John Stewart Mill in Robinson et
al. 2007). Valid causal inference requires comparison of treatment effects among like
populations to overcome the problem of not being able to see the same subjects under both
conditions of treatment and control (Holland, 1986). The validity of causal claims can be
threatened by the effects of history, maturation and selection when a plausible counterfactual
is not available (Campbell, 1979). In the mathematics course-taking literature, selection is the
most challenging of these threats because more prepared students, students with higher levels
of family income, male students, and students from certain race or ethnic groups are more
likely to be placed in, enroll in, and complete college mathematics courses, as well as, complete
math-intensive degrees (Jacobs, 1996; Parker, 2005). Standard regression techniques on the
impact of math course-taking on later educational outcomes that do not address these factors
will produce biased results.
The prior literature relies primarily on OLS regression, logistic regression, and eventhistory analysis methods (Adelman, 2005; Herzog, 2005; Roska & Calcagno, 2008; Calcagno,
Crosta, Bailey, & Jenkins, 2007). Without random assignment to treatment or random
sampling, these techniques have a limited ability to control for all preexisting differences
between students (Calcagno et al, 2007). Student and family characteristics are correlated with
math registration and enrollment, but also selection into colleges and majors (Ferguson, 2009).
Students with higher levels of academic preparation or who come from more advantaged
backgrounds tend to attend more selective colleges and to cluster in science and mathintensive majors (Adelman, 2005, 2006; Bailey & Alfonso, 2005; Cabrera et al., 2005).
Cullinane, College Math—7
Without accounting for these factors, the impact of math on outcomes may be driven by
course-taking patterns at more selective institutions, which also have higher graduation rates.
Few studies that use system-level or state-level datasets control for between-institution
variation in math course-taking patterns and various educational outcomes (Calcagno et al.,
2007; Adelman, 2006); however, there are single-institution studies, such as Herzog’s 2005
study, that suggest the impact of mathematics is not solely a between-institution phenomenon.
The impact of math course-taking may also be confounded by student major—a factor that
affects GPA and other outcomes, as well as, whether a student takes math in the first semester.
The endogenous role of major is largely unaccounted for in the higher education literature,
although there is evidence of its importance in many economic returns-to-education studies.
For example, Paglin and Rufolo (1990) find that 82 percent of the variance in entry-level wages
across majors can be explained by average math GRE scores. Rose and Betts (2004) find that
more advanced math courses in high school produce higher earnings using both OLS and an
instrumental variables approach, while Levine and Zimmerman (1995) use OLS in their study of
how math and science course-taking affects major. They find that taking more credits of math
increases the probability of selecting a STEM major. These effects are not unbiased due to
selection problems, but do contribute to the evidence-based rationale for including major in
the current study.
When correlational studies are misinterpreted as causal for the purposes of
policymaking, unintended policy consequences, like those surrounding K-12 mathematics and
the Algebra for All movement, are likely (Robinson et al., 2007, Clotfelter et al., 2012). The
Algebra for All movement was based on research that was interpreted to mean advanced
Cullinane, College Math—8
mathematics courses and earlier course-taking increase student achievement (Smith, 1996).
Despite the fact that studies were based on the practices of schools placing more successful or
higher ability students in more advanced math courses earlier, major policy changes were made
to place more students, many of them less academically prepared students, into Algebra in the
8th grade (Loveless, 2008). Newer evaluations have demonstrated that conflating the
association between math and later success with a causal interpretation has produced
unintended outcomes including higher rates of failure, drop out, lower grades, and that
students affected by the policy were not likely to outperform peers in states without Algebra
for All policies nor attend college at higher rates (Clotfelter et al., 2012; Allensworth, 2009;
Loveless, 2008).
Mathematics literature has consistently found mathematics course-taking to be
positively, substantively, and statistically significant predictor of a variety of postsecondary
educational outcomes at both two-year and four-year institutions. The topic is ripe for further
research, in particular, by designing studies that can help identify causality as a basis for
policymaking to support degree completion and by addressing the important role of major in
influencing outcomes. To do so requires comparing like populations, most satisfactorily
accomplished through randomized-control trials. While randomized control trials are largely
impractical for the question at hand, we can take advantage of natural experiments where
“assignment” to treatment and control stem from an exogenous factor. In the next section I
describe just such a research design which relies on class size limitations to “assign” students
randomly to treatment and control.
Cullinane, College Math—9
Design
As discussed above, prior studies have made claims that early success in college math
courses increases the likelihood of degree completion. While the literature has convincingly
asserted there is a relationship between math completion and degree completion, it has been
unable to differentiate the selection effects from treatment effects. The challenge is that
students who are more prepared for college mathematics, more intrinsically motivated, or
whose selected major requires more numerous or advanced math are more likely to take math
early, more likely to complete it, and also more likely to complete a degree.
The current study will address the non-random selection of students into college
mathematics courses through the use of a regression discontinuity design. Regression
discontinuity is a quasi-experimental research design used when assignment to treatment and
control takes place on the basis of a numeric threshold or cut-off point. This so-called forcing
variable in my regression discontinuity design will be class size limitations in college
mathematics courses. The forcing variable serves as an exogenous source of variation and
random assignment to treatment that does not directly affect the outcome of interest, in this
case, degree completion. Using the forcing variable enables identification of the unique effect
of early math course taking (Bloom, 2009).
Assignment to treatment is essentially random under these conditions and I can assume
students in the control group and students in the treatment group are equivalent in all
observed and unobserved characteristics. Assignment to treatment occurs during the course
registration process. There are fewer seats available in entry-level mathematics courses than
there are students interested in enrolling for them. The system of registration at the university
Cullinane, College Math—10
permits enrollment on a first come, first serve basis at assigned times. Once the flood gate
opens, registration takes place extremely rapidly, often within the first day or first couple of
hours. Students that attempt to enroll and are successful are no different than students that
attempt to enroll and are unable to do so because of space limitations in a class. Students who
are unable to register may be placed on a waitlist and later allowed to enroll, placed on a
waitlist and later denied access, or may be denied access entirely at the time of attempted
registration. For the purposes of this study, students who successfully enrolled in the course
(via waitlist or directly) will be compared to students who were waitlisted and denied
enrollment or directly denied enrollment in the first semester. These control students may take
math in a subsequent semester or may not ever enroll in a math class.
Enrollment limitations vary across different math courses and sections, so data will be
centered around the class size cut-off, regardless of whether the course serves 50 students or
300 students. Within a small window of time (perhaps the first day of registration) or a small
range of students (say the first 5 students on the waitlist compared to the last 5 students
enrolled in the course), I can make a valid contrast between treatment and control populations
(Bloom, 2009).
Possible limitations are that particularly motivated or persistent “control” students may
successfully enroll in a different section or course than they first attempted to enroll and for
which they were either waitlisted or unable to enroll. These are the “always takers” Bloom
(2009) describes. There may also be differences among students who are taking College
Algebra and those taking Calculus with Differential Equations in terms of prior academic
coursework. Also, the true effect of early math course taking may be obscured by students
Cullinane, College Math—11
taking math, either through dual enrollment or transferred from another institution of higher
education, in places other than the institution in the study. These students will not appear in
my study. Neither will students who would like to register or intended to register but see a
class and waitlist are already full. This student may not even attempt to register, thus they will
not appear as they should in the control group.
To address these limitations, I will systematically examine the registration patterns of
enrollees to determine whether students that were denied access in a course and subsequently
enrolled in a different course or section systematically differ from students that were denied
access and delay math course enrollment until a later semester. I will further check for the
robustness of my findings by examining high level and lower level math courses separately and
comparing these to the average effect of early math course-taking on graduation. Finally, I will
control for whether a student has received credit by exam or transfer credit prior to enrolling as
a freshman.
Data
This study uses 10 years (2001-2010) of administrative, admissions and financial aid data
from a large public institution in Texas, which includes pre-college characteristics identified
during the admissions process and college-level student characteristics and experiences. The
number of observations is 18,585. This study focuses on freshman students who entered the
university between AY 2001-2002 and AY 2003-2004. Data are limited to these years to
examine 4-year and 6-year graduation rates. The unique feature of this dataset is the robust
registration information which captures not only successful enrollment but unsuccessful
attempts to register for a course and the exact time of attempted registration.
Cullinane, College Math—12
The key variables in this study are student attempts to register for a math course in the
first semester, whether a student enrolled in a math course in the first semester, and class size.
First semester freshman in the sample took 38 unique math courses, 11 of which are very high
enrollment and serve 94 percent of all freshman in the entering classes in 2001, 2002 and 2003.
Attempting to register is a dummy variable coded 1 if a student tried to enroll in a math course
in the first semester. Enrollment is a second dummy variable that is coded 1 if the student
successfully enrolls in the course. Class size is a transformed continuous variable where the
maximum class capacity is centered at 0.1 Finally, there is another dummy indicator to denote
whether registration for the course met or exceeded capacity. Only courses that exceeded
capacity will provide sufficient data to conduct the regression discontinuity and will be included
in the analysis. Interviews with registrar staff suggest Calculus I and II are oversubscribed every
fall and particularly well-suited to the analysis.
Outcomes variables include first year GPA, credits accumulated, graduation and time to
degree. First year GPA is a cumulative total for the first two semesters of college in residence
at UT, adjusted for student major. Credits accumulated measures the total number of college
course credits a student has accumulated after four and six years. A dummy variable equal to 1
indicates graduation if the student completes all requirements and graduates. Time to degree
is calculated as the likelihood of graduating in 4 years, also coded with a positive dummy
indicator.
1
An alternative representation of class size is course registration time, centered at the time maximum enrollment
capacity was reached. I will examine class size first as the number of students because I believe it to be more easily
interpretable, but will check the stability of estimates and the appropriateness of bandwidths using both measures of
class size.
Cullinane, College Math—13
Finally, I include a host of control variables. In theory, the students in treatment and
control groups should be statistically indistinguishable from one another based on the as-if
random assignment of class size limitations in college math courses. To check whether
assignment is random or near random, it will be necessary to compare observable
characteristics of students in the two groups. In addition, even when
assignment is random, sometimes there are factors that by chance differentiate treatment and
control groups. For this reason, I will also include covariates in the subsequent analysis.
Covariates include whether a student completed the recommended high school courses, SAT
score (or ACT converted to SAT score), gender, percentile rank of high school graduation, race,
mother’s education, family income, unmet financial need for college and major. The
recommended high school program indicates students completed four years of English,
mathematics, science, and social studies in high school. Percentile rank of high school
graduation is calculated by high
school rank divided by graduating class size. Racial groups include white, black, Hispanic, Asian,
and other. Mother’s education is a categorical variable that indicates whether a student’s
mother had less than a high school degree, a high school degree, some college, a bachelor’s
degree, or a master’s or higher level education. Family income is a categorical variable that
identifies annual incomes less than $20,000, between $20,000 and $40,000, between $40,000
and $60,000, between $60,000 and $80,000, between $80,000 and $100,000, and more than
$100,000. Unmet need is a continuous variable that indicates the amount of financial resources
remaining net of expected family contributions and financial aid.
Cullinane, College Math—14
Table 1:
Cullinane, College Math—15
Summary statistics are included in table 1. Column 1 provides summary data for all
students in the analytic sample. Columns 2 and 3 describe the characteristics of students who
took a first-semester math course and those who did not. More detailed information on
Cullinane, College Math—16
mathematics course-taking is listed in table 2, including number of unique math courses, total
enrollment, and percent of students taking math in the first semester.
First Semester Mathematics Courses (2001-2002 AY - 2003-2004 AY)
Enrollments Proportion
Differential and Integral Calculus
Elementary Functions and Geometry
Sequences, Series, and Multivariate Calculations
Differential Calculus
Introduction to Mathematics
Calculus I For Business & Economics
Seminar Course
Applicable Mathematics
Calculus II For Business & Economics
Integral Calculus
Conference Course
Emerging Scholars Seminar
TOTAL
Cumulative
Proprotion
3,984
28%
28%
2,557
1,967
1,574
1,561
595
387
355
234
234
217
207
18%
14%
11%
11%
4%
3%
2%
2%
2%
2%
1%
46%
59%
70%
81%
85%
88%
90%
92%
94%
95%
97%
14,348
Table 2
Methods
I estimate the effect of early math course-taking on first-year GPA, credits accumulated,
graduation and time to degree using graphical analysis, differences in means tests, and
regression techniques. I will begin by using graphical analysis of student participation based on
class size2 (x-axis) and the four educational outcomes of interest (y-axis). First, I confirm
student assignment to treatment follows sharply from the class size forcing variable. I then
examine the relationship between the enrollment limit and the outcomes to determine
whether the assumption of continuity of conditional regression functions is met and whether
2
Or registration time.
Cullinane, College Math—17
model specification is linear, low-order, or higher order polynomial (Imbens & Lemieux, 2008).
Finally, I examine the differences in means at the point of discontinuity for treatment and
control. I hypothesize early math course taking will be negatively related to freshman GPA and
positively related to college completion. I suspect it will also decrease time to degree and the
number of accumulated credits. Effects will appear graphically as a separation between
predicted trend lines on either side of the discontinuity (Imbens & Lemieux, 2008). This is the
average effect of intent--to--treat at the cut-point (ITTC) (Bloom, 2009).
I use logistic regression to model the binary outcome of graduation and OLS to model
the continuous outcomes of freshman GPA, credits accumulated, and time to degree (Calcagno
& Long, 2008). Assuming a sharp regression discontinuity at the class size limitation, I examine
a basic model, then add student covariates to control for any lingering differences between
treatment and control students. Standard errors are clustered by course. I will assess the fit of
lower-order and higher order polynomials in model specification and use a non-parametric
kernel smoother in the case of nonlinearity (Calcagno & Long, 2008). Finally, I will examine
variable bandwidths both for the number of students near the cut off as well as for a limited
period of time to see whether my results are sensitive to these changes (Imbens & Lemieux,
2008).
Without access to the detailed registration data at the moment, I cannot yet tell if the
relationship between the earliest registrants will be different from the later registrants. If the
registration process takes place more slowly, it is possible I will see a general negatively sloping
trend in the relationship as in Example Figure 1. If the registration process takes place very
quickly, within the first day of registration, there may be no relationship between the order in
Cullinane, College Math—18
which students register for a particular course and their later outcomes (Example Figure 2)
(Calcagno & Long, 2008). Each alternative is described graphically below in its hypothesized
relationship with freshman GPA.
The nature of the relationship between order of registration and outcomes will
influence whether assignment is as-if random at the point of discontinuity only or whether the
registration process is more akin to a randomly selected lottery. If the latter is true, a
regression discontinuity design may not be necessary and a simple differences in means test
would provide statistically rigorous results. This determination will be made after examining
the registration data.
Example Figure 1:
Example Figure 2:
Cullinane, College Math—19
Freshman
GPA
Class Size
Class Size
Results
This section includes the descriptive statistics of the impact of math course-taking on
college outcomes. Because math courses are challenging, I expect in the short run that early
math course-taking may decrease freshman GPA. Over the longer term, I expect that
completing math would clear students of a key hurdle in their academic journey, thus the early
math completers are also more likely to complete college than those that avoid or delay math,
with shorter time to degree and fewer total credits accumulated. Students that are able to
directly enter key freshman courses required for entrance into a major are more likely to satisfy
all graduation requirements in a more timely manner. Students delayed from entering math
courses in their first semester may need to wait a full year before certain classes are available
again, thus slowing time to degree and increasing extraneous credit accumulation.
First I examine the descriptive statistics and graphical analysis. I do not yet have access
to registration data, so the descriptive statistics are limited to showing the differences between
Cullinane, College Math—20
all students who took math in their first semester and students who did not take math in their
first semester. I do not expect these students to be similar, and in fact we find that students
that do not take math in their first semester have slightly higher GPAs, higher graduation rates,
and more accumulated credits. These students graduated lower in their high school graduating
classes (78th percentile compared to the 81st percentile) and are much more likely to be female
(64 percent versus 48 percent). Asian students are more likely to take math in their first
semester. Students that do not take math in their first semester have higher levels of parent
education and family income. Most of these results conflict with my prior expectations and the
prior literature on this topic.
These pooled statistics mask key differences between students that begin in more
advanced math courses relative to students that begin their college careers in more elementary
math courses. In Table 2, I present graduation rate data, broken down by math course, using
the five highest enrollment first-semester math courses to illustrate the phenomenon. The
courses are listed in ascending order of difficultly. Elementary Functions and Geometry is the
lowest level course and students that begin their math course-taking at this level have the
lowest graduation rate—72 percent. As math courses advance, we see that graduation rate
generally increases with each step. Students that took Introduction to Mathematics graduated
at a rate of 76 percent, Differential Calculus at 81 percent, Differential and Integral Calculus at
76 percent, and Sequences, Series, and Multivariate Calculus at 84 percent. Accounting for just
these five courses, the data reveal a more expected relationship between mathematics coursetaking and outcomes for students that did not take math in the first semester. Students who
did not take math graduated 81 percent of the time, compared to 84 percent for students in all
Cullinane, College Math—21
Seminar Course
0.01
0.09
0.13
0.05
0.22
Applicable Mathematics
0.10
0.31
0.14
0.22
0.41
Calculus II For Business & Economics
0.22
0.41
0.18
0.36
0.48
Integral Calculus
0.44
0.50
0.33
0.23
0.42
Conference Course
0.22
0.41
0.19
0.12
0.33
Seminar
0.01
0.10enrollment
0.02courses listed.
0.02 At this
0.13
other
firstCourse
semester math courses besides
the top five
point,
Emerging Scholars Seminar
0.053
0.223
0.059
0.040
0.197
it appears that which math course students take first may signal prior academic preparation or
other factors that are associated with graduation in ways that fit my prior expectations and
align with previous studies.
Table 3:
Graduation Rates by First Semester Mathematics Courses (2001-2002 AY - 2003-2004 AY)
Graduated
Total
Graduation
No
Yes
Rate
No First Semester Math
1,300
5,453
6,753
81%
Elementary Functions and Geometry
662
Introduction to Mathematics
601
Differential Calculus
288
Differential and Integral Calculus
934
Sequences, Series, and Multivariate Calculations
311
1726
1933
1198
2920
1613
2388
2534
1486
3854
1924
72%
76%
81%
76%
84%
Other First Semester Math Courses
2,074
2,489
83%
415
The next two issues I need to check is whether some of the most advanced or most
advantaged students do not take math in their first semester because they complete AP or dual
enrollment courses before matriculating to the university. Students that have already satisfied
college mathematics course requirements are likely very different from students who need to
take math but avoid it. AP course participation and dual enrollment are rapidly growing in
Texas, and it may be the case, the strongest students or students who come from very high
quality high schools do not need to enroll in mathematics during their first semester, or ever, in
college. The unexpectedly lower high school rank among the students who do not take first
semester math seems to substantiate this hypothesis. The top ten percent law in Texas
Cullinane, College Math—22
guarantees automatic admission to the state’s public universities for students graduating in the
top ten percent of their class. Prior research on high school quality in Texas using the top ten
percent law indicates that high quality schools tend to matriculate students with slightly lower
high school ranks, while low quality tend to matriculate students with higher high school rank
(Black, Lincove, Cullinane & Douglas, 2012 forthcoming). At this point, the data on credit by
exam is not differentiated by level, year, or location in ways that allow me to differentiate dual
enrollment, AP, and other forms of test-based credit without additional information from the
university. This will be an important step in the subsequent analysis.
The final issue is around major. I am interested in examining whether students that do
not take math in their first semester may be concentrated in non-STEM majors that may also
have higher graduation rates. Again, the current data does not permit this analysis readily.
Some students are admitted to majors as freshman, others must apply and be accepted to a
major after completing a series of required courses. The timing therefore is not uniform, nor is
the major declaration process consistent across majors. I also encounter a “success bias” in
that students with information about a declared or intended major are likely more successful
than the many students for whom I have no information about choice of major. I am exploring
the possibility of totaling STEM credits accumulated in the first year as a proxy for
concentration or intended major.
Without access to the necessary registration data, I am unable to complete my
examination of the effects of math on each of the four outcome variables as identified through
graphical and regression techniques, first using basic models and then adding student controls.
Cullinane, College Math—23
The future analysis will be limited in its generalizability because inferences can only be
made to students who are motivated or interested to take math in their first semester. The
findings in this paper will tell us little about the effectiveness of early math on college outcomes
of students who satisfy college math requirements in high school or who are unmotivated or
uninterested in taking math early in college (Bloom, 2009).
Conclusion
At this point it is premature to make definitive claims about my results. At a minimum, my
review of the prior literature should raise concerns about the robustness of findings and about
making policy on the basis of correlation studies. The initial trends I see in the descriptive data
raise questions about average math completion and its predicted causal impact on ultimate
success as prior studies have asserted. In this university setting, the level of courses seems to
be differentiating student outcomes in more predictable ways. I suspect choice of major and
pre-college math credits are also complicating the story and I will control for these two
important factors in the regression discontinuity analysis.
Whether early math success significantly influences GPA, credits accumulated,
graduation and time to degree are important considerations for university administrators
concerned about organizational efficiency and increasing graduation rates. At this point I
would recommend policymakers do not craft incentives or mandates to influence math coursetaking without valid causal evidence of its effectiveness. While this study is designed to
contribute precisely to this gap in the literature, preliminary examination of the data suggests
math course taking is a complex phenomenon and that policy should not presume causality or
external validity where research cannot support it.
Cullinane, College Math—24
Further research is required to determine whether my anticipated findings reveal
mathematics courses influence later outcomes due to student acquisition of human capital or
signaling. A follow up study that examines the grades of students in subsequent courses that
rely on mathematics may shed light on this question.
Cullinane, College Math—25
References
Adelman, Clifford. “The Toolbox Revisited: Paths to Degree Completion from High School
Through”, 2006. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.108.495.
Allensworth, Elaine, Takako Nomi, Nicholas Montgomery, and Valerie E. Lee. “College
Preparatory Curriculum for All: Academic Consequences of Requiring Algebra and
English I for Ninth Graders in Chicago.” Educational Evaluation and Policy Analysis 31,
no. 4 (December 1, 2009): 367 –391.
Babad, Elisha. “Students’ Course Selection: Differential Considerations for First and Last
Course.” Research in Higher Education 42, no. 4 (2001): 469–492.
Black, S., Lincove, J.A., Cullinane, J., & Douglas, R. (Paper conditionally accepted, revise and
resubmit). Can you leave high school behind? Economics of Education Review.
Bloom, H.S. (2009). Modern Regression Discontinuity Analysis. MDRC Working Paper on
Research Methodology.
Calcagno, J.C. and Long, B.T. (2008). The Impact of Postsecondary Remediation Using a
Regression Discontinuity Approach: Addressing Endogenous Sorting and
Noncompliance. NBER Working Paper No. w14194.
Calcagno, Juan Carlos, Peter Crosta, Thomas Bailey, and Davis Jenkins. “Stepping Stones to A
Degree: The Impact of Enrollment Pathways and Milestones on Community College
Student Outcomes.” Research in Higher Education 48, no. 7 (November 1, 2007): 775–
801.
Clotfelter, C.T., Ladd, H.F., and Vigdor, J.L. (2012). The Aftermath of Accelerating Algebra:
Evidence from a District Policy Initiative. CALDER Working Paper No. 69.
Deming, J.D., Hastings, J.S., Kane, T.J. & Staigler, D.O. (2011). School Choice, School Quality and
Postsecondary Attainment. NBER Working Paper No. 17438.
Federman, M. (2007) State graduation requirements, high school course taking, and choosing a
technical college major. B.E. Journal of Economic Analysis and Policy: Advances in
Economic Analysis and Policy, v. 7, No. 1, pp. 1-32.
Herzog, Serge. “Measuring Determinants of Student Return Vs. Dropout/Stopout Vs. Transfer: A
First-to-Second Year Analysis of New Freshmen.” Research in Higher Education 46, no. 8
(December 1, 2005): 883–928.
Holland, P. W. (1986), Statistics and causal inference. Journal of the American Statistical
Association, 81, 945-970.
Imbens, G.W., Lemieux, T., Regression discontinuity designs: A guide to practice, Journal of
Econometrics (2007), doi:10.1016/j.jeconom.2007.05.001
Jacobs, J.A. (1996). Gender Inequality and Higher Education. Annual Review of Sociology
Vol. 22, (1996), pp. 153-185.
Kahn, Jeffrey H., and Margaret M. Nauta. “Social-Cognitive Predictors of First-Year College
Persistence: The Importance of Proximal Assessment.” Research in Higher Education 42,
no. 6 (December 1, 2001): 633–652.
Kane, Thomas J., and Cecilia Elena Rouse. “Labor-Market Returns to Two- and Four-Year
College.” The American Economic Review 85, no. 3 (June 1, 1995): 600–614.
Long, Mark C., Patrice Iatarola, and Dylan Conger. “Explaining Gaps in Readiness for CollegeCullinane, College Math—26
Level Math: The Role of High School Courses.” Education Finance and Policy 4, no. 1
(2008): 1–33.
Moore, C. and Shulock, N. (2009). Student Progress Toward Degree Completion: Lessons from
the Research Literature. Institute for Higher Education Leadership and Policy: California
State University Sacramento.
Ma, Xin, and Jesse L. M. Wilkins. “Mathematics Coursework Regulates Growth in Mathematics
Achievement.” Journal for Research in Mathematics Education 38, no. 3 (May 1, 2007):
230–257.
Parker, Melanie. “Placement, Retention, and Success: A Longitudinal Study of Mathematics and
Retention.” The Journal of General Education 54, no. 1 (January 1, 2005): 22–40.
Robinson, D.H., Levin, J.R., Thomas, G.D., Pituch, K.A., and Vaughn, S. (2007). The Incidence of
"Causal" Statements in Teaching-and-Learning Research Journals. American Educational
Research Journal, Vol. 44, No. 2 (Jun., 2007), pp. 400-413
Roska, J., Calcagno, J. (2008). Making the Transition to Four-Year Institutions: Academic
Preparation and Transfer. Community College Research Center, Teachers College:
Columbia University.
Schneider, B., Swanson, C., and Riegle-Crumb, C. (1998). Opportunities for Learning: Course
Sequences and Positional Advantages. Social Psychology of Education, vol. 2, pp. 25-53.
Smith, Julia B. “Does an Extra Year Make Any Difference? The Impact of Early Access to Algebra
on Long-Term Gains in Mathematics Attainment.” Educational Evaluation and Policy
Analysis 18, no. 2 (June 20, 1996): 141 –153.
Spade, J.Z., Columba, L. and Vanfossen, B.E. (1997). Tracking in Mathematics and Science:
Courses and Course-Selection Procedures. Sociology of Education , Vol. 70, No. 2 (Apr.,
1997), pp. 108-127.
Stiglitz, J.E. (1975). The Theory of "Screening," Education, and the Distribution of Income. The
American Economic Review , Vol. 65, No. 3 (Jun., 1975), pp. 283-300.
Thistlethwaite, Donald L., and Donald T. Campbell. 1960. “Regression--Discontinuity
Analysis: An Alternative to the Ex--Post Facto Experiment.” Journal of
Educational Psychology 51: 309--317 (December).
Cullinane, College Math—27