Summary of Presentation
Mining Real-Time Data to Improve Student Success
in a Gateway Course
by Laurie Iten, Kim Arnold, Matt Pistilli
Eleventh Annual TLT Conference, Purdue University, March 4, 2008
Our presentation told the story of a successful
curricular change in the Department of Biological
Sciences at Purdue University. Before the 2004/05
academic year, one of us (Laurie Iten) made the
recommendation, and her department approved,
that she make a paradigm shift in her approach
to teaching first-year gateway majors’ labs from
“pseudo” experimental/observational traditional
biology lab courses to skills-based lab courses.
Why make such a radical curricular change?
The answer is multifaceted, but it boils down to
our old first-year biology majors’ labs, like those
taught at other large research universities, had
not changed in decades even though they were
ineffective. They were ineffective because of the
challenging logistics of large-enrollment biology
lab courses, and students not having the needed
lab skills, resulting in labs that resembled cooking classes where any experimental/observational
results students obtain were obvious or trivial, thus
making them “flat out” dull and a waste of time.
Laurie wanted a better return for her teaching
investment in beginning biology majors’ labs.
Namely, she wanted better academic performance
out of Purdue’s biology majors, and she wanted to
help retain and graduate more majors. Her initial
“new” labs focused on the following skills students
learned, practiced, and demonstrated proficiency:
quantitative and problem solving skills; handling
cells and tissues, and microscopy; information
and communication skills; and measurements
and solution chemistry. These skills were parsed
into four half-semester lab modules students
took in any order. This year, a change was made
to biology’s first-year major’s laboratory experience. Approximately two and a half of the four lab
modules previously offered were condensed into a
one-semester course. A mini take-home message
from this paradigm shift in teaching first-year
major’s labs is that three and a half years of offering these skills-based labs results in a dramatic
improvement in students’ academic performance
in their subsequent major’s courses, and the department retains more biology majors.
Our presentation also told you about how we
successfully implemented real-time data-mining
analytics in Laurie’s gateway biology lab course
that resulted in even more improved student performance. This initiative’s inception was an ITaP
Brown Bag Seminar presented by John Campbell and Bart Collins, November 2006, entitled
“Student Retention: Mining Student Data to
Support Future Early Intervention Initiatives.”
This year we embarked on a study to determine
the effectiveness of real-time data-mining analytics to improve student academic performance in
Laurie’s first-year major’s lab (see the end of this
summary for the names of all the people involved
in our initiative). As we are well aware, first-year
students typically take large-enrollment gateway
major’s courses where they can receive infrequent
feedback on how they’re doing. Such infrequent
feedback can lead to poor academic performance,
and students leaving a major. Our study’s basic
hypothesis is that early and frequent monitoring
of student performance with warning messages
being delivered to underperforming students will
improve their academic performance, especially in
a first-year gateway course. We are using real-time
data-mining techniques because manually monitoring student performance as a course progresses
is daunting and difficult to do without using
database and data-mining technologies. The datamining analytics we use were developed by John
Campbell here at Purdue (see selected references
at the end of this summary) and they reveal at-risk
students we can notify individually about their
poor performance along with where and how they
can get help. We used Laurie’s first-year biology
lab because of its pedagogical design, and her
use of many of the online asynchronous learning
tools within our university’s course management
system (in conjunction with synchronous face-toface lab instruction), and John’s predicted student
success algorithm uses the analysis of information from our university’s course management
system and student information system. We used
John’s data-mining analytics at the end of each
week’s labs. The output of each weekly analysis
places students into one of three groups based
on a student’s chance of successfully earning an
A or B course grade: high-risk of not getting an
A or B, moderate-risk, and no-risk. We split the
10 divisions of Laurie’s first-year biology lab in
half, and students in 5 of the 10 divisions who
fall in the high-risk and moderate-risk groups
receive intervention email messages along with
personal messages from Laurie or her teaching
staff commencing week two of the semester.
Based on each week’s data-mining report, Laurie
sends weekly intervention messages through the
ninth week of the semester. Students with a high
probability of successfully completing the course
with an A or B in the 5 “test” divisions receive
no intervention messages. Also, the intervention
messages delivered to high- and moderate-risk
“test” division students become progressively
sterner as the semester progresses. The results
of our study from fall semester 2007 are very
promising in that early and frequent intervention
messages result in at-risk students improving their
academic performance, as well as improving their
academic “help-seeking” behavior when compared
to the “non-test” group not receiving intervention
messages. With these encouraging results, we are
continuing this study this semester with some
minor modifications. We are also in the process
of automating the technology, mechanics, and
logistics of our real-time data-mining analytics
of student performance so other faculty here at
Purdue can use this technology with their courses,
especially with their first-year gateway major’s
courses. Finally, the results of our real-time datamining study with Laurie’s first-year biology
lab, as well as, the detailed analysis of student
academic performance before and after she made
the paradigm change in her approach to teaching first-year biology majors’ laboratories will be
published at the end of this academic year.
Real-time data-mining team for student success
Laurie Iten
Associate Professor, Department of Biological Sciences
liten@purdue.edu
Kim Arnold
Educational Assessment Specialist, ITaP TLT
kimarnold@purdue.edu
Matt Pistilli
Assistant Director, Student Access, Transition, and Success Programs
mdpistilli@purdue.edu
Tim Kerr
Academic Advisor, Department of Biological Sciences
kerrtp@purdue.edu
Ashley Jones-Bodie
Graduate Research Assistant, ITaP TLT
abodie@purdue.edu
John Campbell
Associate Vice President Rosen Center for Advanced Computing, ITaP
john-campbell@purdue.edu
Selected relevant publications of John Campbell
Campbell, J. P. & Oblinger, D.G. (2007) Academic Analytics. EDUCAUSE (http://connect.educause.
edu/Library/Abstract/AcademicAnalytics/45275)
Campbell, J. P. & Rud, A.G. (Submitted) Exploring the Ethical Issues Related to Technology, Analytics
and Academic Success: Obligation of Knowing. Journal of Educational Computing Research.
Campbell, J. P. (2008). Utilizing course management data to predict undergraduate student success.
Journal of College Student Retention, 9(3): 186.
If you want to hear more about our real-time data-mining project, Laurie Iten will be giving the ITaP
Brown-Bag Seminar, April 2, 2008, 12:30-1:30 PM in STEW 318, on our project that will include
more pedagogical details, and more results from our study.
Summary of Mining Real-Time Data to Improve Student Success in a Gateway Course