Int'l Conf. Frontiers in Education: CS and CE | FECS'15 |
215
Flipping a Data Structures and Discrete Mathematics
Class
Anja Remshagen
Computer Science Department, University of West Georgia, Carrollton, Georgia, US
Abstract - This paper investigates the impact of the flipped
classroom in a data structures and discrete mathematics
course. The retention rates and students’ grades when the
flipped classroom model was applied in the course are
compared with the retention rates and grades in the
traditional lecture model. A two-week study was performed to
compare the students’ performance in the two different
pedagogical models. Results are mixed and don’t show a
clear performance improvement in favor of the flipped
classroom. However, grades and retention rates were
significantly higher in the flipped classroom, indicating that
the flipped classroom might help raising the confidence level
of students.
Keywords: computer science education, flipped classroom,
traditional lecture, data structures, discrete mathematics
1
Introduction
Data Structures and Discrete Mathematics I (CS3151) is
a required course to complete the Bachelor of Science in
Computer Science at the University of West Georgia. The
University of West Georgia is a regional, public institution.
Most undergraduate students are traditional students that have
started college right after finishing high school in the West
Georgia region. Many of these students are first-generation
college students. The racial demographics of the students is
similar to the racial demographics of Georgia. For example,
in the academic year 2013/2014, 33.5% of the students were
Black/African Americans and 56.3% were Caucasion/White.
In the course CS3151, Data Structures and Discrete
Mathematics I, computer science majors learn about
elementary data structures and algorithms and about
fundamental concepts in discrete mathematics. They
implement and apply the data structures and related
algorithms in small-scale Java programs. The course is
typically taken by computer science majors after they have
completed the introductory computer science sequence CS1
and CS2, which introduces students to the object-oriented
programming language Java. As one of the first courses of
theoretical and abstract nature, many students do not feel
prepared for the rigor of CS3151. This is reflected by a high
withdrawal rate. For example, in Fall 2012, out of 44 students
11 students withdrew from the class. In Spring 2013, out of
36 students 6 students withdrew. Some of the withdrawing
students change their major; others attempt the class a second
time.
To improve the retention rate in CS3151, the flipped
classroom had been applied in Spring 2014. The flipped
classroom is a pedagogical model where students learn new
content on their own time outside the classroom and apply the
material in problems and hands-on-activities during class
sessions [5], [6]. The flipped classroom has been explored in
several computer science disciplines, including introductory
computer science courses [1], [7], computer architecture [4]
and software engineering [3], to name only a few. Most
studies demonstrate more engagement and enthusiasm among
the students. Some studies show that there was no
performance improvement [1], [4], or they had not yet
sufficient data to study the students’ performance in a flipped
computer science course [3], [7]. This paper compares the
students’ performance in the flipped classroom with the
performance in the traditional lecture model during a twoweek study. It also explores retention rates and changes in
grades among the recent course offerings.
2
The Flipped Classroom in CS3151
Before the introduction of the flipped classroom in
CS3151, the class followed the traditional lecture model. Two
issues that seem to impede students’ success came up
repeatedly when the traditional lecture model had been
applied: (1) Students asked for more practice problems. (2)
Some of the weaker students needed significant help to get
started on the programming exercise. In Spring 2014, the
flipped classroom was implemented to help students with the
two issues addressed above and, in turn, to improve retention
rates. As the current generation of students is used to
consume visual media, video recordings for the course
material were produced in the form of narrated slides, instead
of assigning readings from a textbook. The students were
required to watch the videos before class. In class, students
worked through exercises individually or in teams of two.
After working through one or two exercises, typically a
student presented his or her solution. Sometimes the
instructor worked out the solution with the students on the
whiteboard.
As in preceding semesters, every week homework was
assigned in Spring 2014. Most of the homework exercises
216
Int'l Conf. Frontiers in Education: CS and CE | FECS'15 |
were very similar to the exercises worked out in class and
contained a mix of exercises at different levels of Bloom’s
taxonomy. In addition to written exercises, in many
classroom meetings the students worked on a programming
exercise that required the implementation or the application
of a data structure and its associated algorithms. During the
class meetings, students could program in teams of two. The
programming exercises typically could not be finished in
class due to time-constraints. Thus the completion of the
programming exercises was assigned as part of the
homework. Students had to complete all homework
individually. At the end of the time reserved in class for the
hands-on exercise, ideas of how to complete the
programming task were discussed. This gave the students
with weak programming skills a good start on programming
exercises.
3
Table 1. Schedule of interventions.
Wednesday (Mar 24)
Pretest taken by Group L and F
Assignment of videos only
accessible to Group F
Monday (Mar 31)
Lecture for Group L
Assignment of homework to
Group L
Wednesday (Apr 2)
Flipped classroom meeting for
Group F
Assignment of homework to
Group F
Monday (Apr 7)
Posttest taken by Group L and F
Homework due for Group F and
Group L
Study and Data Collection
In order to determine whether students had a better
understanding of the material, the concepts, and of the
algorithms by using the flipped classroom, the class was split
into two groups for one week. Group L was exposed to the
traditional lecture model during that week, and Group F to
the flipped classroom. Both groups were taught exactly the
same material. Group L met on a Monday for a traditional
lecture that introduced the abstract data type priority queue,
the heap data structure, its associated operations, and heap
sort. The same material that was covered in the lecture was
recorded through narrated slides. The recordings were
assigned to Group F to watch before the class meeting on
Wednesday. The recordings were not available to Group L.
Group F worked through exercises in their class meeting. A
pre- and posttest assessed the progress of each group. Table 1
displays the schedule for this study.
Different problems were tackled by the students in
examples during the lecture, in exercises for the flipped
classroom, in a homework assignment, and in the pre- and
posttest. We can split all these problems into four categories
outlined in Table 2. The first category (KNOW) required
students to understand concepts and recall related facts. These
problems are located at the first and second level of Bloom’s
Taxonomy. The second category (PRP) required students to
interpret the representation of heaps and heap properties.
These problems are at the second and third level of Bloom’s
Taxonomy. The third category (ALG) demanded students to
perform and trace the algorithms that are executed by the
operations on priority queues and heaps. These problems are
at the third level of Bloom’s Taxonomy. The last category
Table 2. Problem categories.
Category
KNOW
PRP
ALG
OTHER
Description
General knowledge
about heaps and
priority queues
Properties of
heaps and their
representation
Operations on
heaps and priority
queues
Includes problems
at the Bloom’s
synthesis level
Examples in lecture
for Group L
x
x
Exercises in class
for Group F
x
x
x
x
x
Homework for
Group L and F
Pre/Posttest for
Group L and F
x
x
x
Int'l Conf. Frontiers in Education: CS and CE | FECS'15 |
217
(OTHER) included problems at higher Bloom levels. These
problems were not suited for a quiz-like test that can be
completed in a few minutes. Thus only the first three
categories were assessed in the pre-and posttest. Table 2
displays the occasions at which the students had been
exposed to problems of the corresponding categories.
KNOW
Table 3. Problem categories.
ALG
PRP
Students in Group L were occasionally asked problemsolving questions during the lecture, but participation was
voluntary. The recorded slides for Group F did not contain all
the examples that were worked out on the whiteboard as part
of the lecture. Instead, Group F worked through related
exercises on Wednesday. In addition to problems of category
ALG and PRP, Group F worked through a problem of
category OTHER.
To assess the difference in performance, a pretest and
posttest were administered in the class meeting in the week
before, respectively after, the groups met separately. The preand posttest contained eight questions as shown in Table 3.
The questions are split into the three categories KNOW, PRP,
and ALG.
Q1
Determine whether a heap is a suitable data
structure to implement a waiting line
in discrete event-driven simulation.
Q2
Identify the concept that underlies heap sort.
Q3
Identify the time complexity of heap sort.
Q4
Determine the arrays representation of a heap.
Q5
Determine whether a given tree is a heap.
Q6
Trace a given sequence of operations on a
priority queue.
Q7
Trace the insert operation on a given heap.
Q8
Trace the remove operation on a given heap.
problems of category OTHER. Hence not only the students in
Group F, but also the student in Group L, that followed the
lecture model, had worked on exercises themselves as well.
Programming exercises were not assigned in that week.
The goal of the assessment is to determine whether
students learn better through the use of the flipped classroom.
In the lecture model learning takes place mostly outside of the
classroom. In CS3151, many students delay studying the
class material until they have to complete a homework
assignment. Hence to fairly compare the learning in both
models, all students had to complete the same homework
assignment before the posttest was given. The assignment
consisted of two problems of the category ALG and of two
4
Results
Table 4 shows the results of the pre- and posttest. The
maximum points for each question were 1.25. The mean and
the standard deviation is displayed for each test question for
Group L and F. The column labeled DIFF displays the
difference among the means of the pre- and posttest.
Table 4. Results of the pre- and posttest.
Group L
ALG
PRP
KNOW
Pretest
Posttest
Group F
Diff
ES
MEAN
SD
MEAN
SD
Q1
0.63
0.66
0.25
0.53
-0.38
Q2
0.88
0.60
1.00
0.53
Q3
0.50
0.65
0.63
Q4
0.00
0.00
Q5
0.41
Q6
Pretest
Posttest
Diff
ES
MEAN
SD
MEAN
SD
-0.44
0.42
0.65
0.42
0.65
0.00
0.00
0.12
0.16
0.42
0.65
0.83
0.65
0.41
0.46
0.66
0.13
0.14
0.00
0.00
0.42
0.65
0.42
0.65
1.13
0.40
1.13
2.85
0.21
0.51
0.63
0.68
0.42
0.49
0.49
1.13
0.40
0.72
1.14
0.42
0.65
1.20
0.127
0.78
1.19
0.50
0.65
1.13
0.40
0.63
0.83
0.83
0.65
1.04
0.51
0.21
0.25
Q7
0.25
0.53
1.13
0.40
0.88
1.33
0.21
0.51
1.04
0.51
0.83
1.15
Q8
0.25
0.53
0.63
0.66
0.38
0.44
0.42
0.65
1.04
0.51
0.62
0.76
218
According to Table 4, the lecture group showed gains in
some areas while Group L showed gains in others. We
discuss the results for each category. Both groups did not
perform very well on the questions in the first category
(KNOW). This is not very surprising, as the questions in this
category had not been practiced in the homework or in one of
the class meetings in any form. Group F shows a stronger
improvement than Group L. It seems that the students were
better able to retain the knowledge through the posted videos
than through the lecture. Possibly, the students in Group F
profited from the advantage that they could review the videos
repeatedly if they could not follow the presented material the
first time.
Recall that the lecture group had seen examples and the
flipped classroom group had worked through exercises
related to the questions in the second category (PRP). But
there had been no homework exercises on the corresponding
topic. Group L improved significantly in the first question of
category PRP, while Group F improved only slightly. Both
Groups showed a similar improvement in the second
question, where the improvement was slightly stronger for
Group F.
In the last category (ALG), Group L improved more
than Group F in the first two questions, and Group F
improved more in the last question. Recall that both groups
had seen examples and exercises, respectively, of the
corresponding topic. Both groups had seen questions in the
homework assignment that were very similar to the questions
in the pre-and posttest. These questions practice operations
on priority queues and heaps. The complexity of the
operations in Q6, Q7, and Q8 increases with Q6 covering the
least complex operations and Q8 covering the most complex
one. It seems that the hands-on exercises in the flipped
classroom helped the students learn in particular the more
complex the problems better. Another reason for the better
performance on complex tasks might be that students in the
flipped classroom had gone through the correct solution to
similar exercises in the class. For simpler exercises most
students are able to determine the correct solution on their
own. But for complex ones, students may need to see more
solutions of related problems.
We also compare the final grades of students in Spring
2014 with the grades in preceding semesters when the
traditional lecture model had been applied. Before Spring
2014, CS3151 had been offered in Spring 2013, Fall 2012
and Fall 2011 and had been taught by the same instructor as
in Spring 2014. There has been significant improvement
concerning the withdrawal and passing rates. In Spring 2014,
81.5% of the students passed the class with a C or better. The
passing rates in the preceding semesters were 66.7% (Spring
2013), 63.6% (Fall 2012), and 53.7% (Fall 2011),
respectively. However, it is hard to compare final grades as
the grade calculation has differed in different semesters. For
example, the grades from Fall 2011 to Spring 2013 included a
Int'l Conf. Frontiers in Education: CS and CE | FECS'15 |
quiz grade that was determined by the use of iClickers. Also
the grading rubric for homework had been changed in Spring
2014, typically resulting in higher grades. Thus, it is hard to
determine if the students’ performance has indeed improved,
and if so, in how far the improvement is based on the flipped
classroom model.
Table 5 lists for each semester and for each final letter
grade the number of students achieving that letter grade and
the corresponding percentage of students with that letter
grade in the class. For example, in Spring 2014 four students
received an A in the class, which is 14.8% of the students.
This is the highest percentage of students that have earned the
letter grade A compared with the other class offerings. No
student withdrew from the class in Spring 2014 while about
17% to 25% withdrew from the class in preceding semesters.
Table 5. Grade distribution per semester.
Class
size
A
B
C
D
F
W
Fall
2011
Fall
2012
Spring
2013
Spring
2014
54
44
36
27
3
(5.6%)
14
(25.9%)
12
(22.2%)
10
(18.5%)
4
(7.4%)
1
(2.3%)
15
(34.1%)
12
(27.3%)
4
(9.1%)
1
(2.3%)
2
(5.6%)
5
(13.9%)
17
(47.2%)
2
(5.6%)
4
(11.1%)
4
(14.8%)
6
(22.2%)
12
(44.4%)
2
(7.4%)
3
(11.1%)
11
(20.3%)
11
(25.0%)
6
(16.7%)
0
(0%)
Considering that no student withdrew from CS3151 in
Spring 2014, the flipped classroom seems to strengthen the
students’ confidence level, which has also been confirmed by
[2], for example. One might argue that the higher grades
might be the cause for the low withdrawal rate. Probably the
grades of a student play an important role in the decision to
withdraw from a course. But this is likely not the only cause
since in previous semesters many student withdrew from the
course even though they still had passing grades.
As pointed out before, the letter grade does not offer a
true comparison as far as mastering the course content is
concerned. A better indicator whether students perform better
might be the test grades. In all semesters very similar tests
were given. Table 6 lists the total average test grade. In each
semester, except for Fall 2011, three tests were given. In Fall
2011, four tests were given. The maximum test grade for each
Int'l Conf. Frontiers in Education: CS and CE | FECS'15 |
test is 100. The test grades do not demonstrate a clear
performance improvement. Note the total average includes
only the test grades of students that had not withdrawn from
the course. Thus the calculated average may favor semesters
at which more students withdrew.
Table 6. Total average tests grades per semester.
Semester
5
Average total test grade
Spring 2014
72.07
Spring 2013
67.81
Fall 2012
73.95
Fall 2011
71.18
Conclusions
Based on the two-week study and on test grades, it does
not seem that students are performing better overall on
exercises in the flipped classroom compared to the traditional
lecture model. The performance was compared after the
corresponding material had been covered and practiced in
both classroom models. There is some indication that the
students perform better in the traditional classroom on less
complex problems for which they have seen examples in
class. Students may perform better on complex problems in
the flipped classroom if those problems have been practiced
in class. The final grades have been significantly better in the
flipped classroom. However, a comparison among different
semesters is difficult due to the different student population
and the difference in grade calculation. No student withdrew
from the flipped classroom while about 17% to 25% of the
students had withdrawn from previous course offerings that
applied the lecture model. This may be contributed to the
higher grades students received before the withdrawal
deadline. But the flipped classroom may also raise the
219
students’ confidence level as not all withdrawals are made by
students with failing grades.
6
References
[1] Campbell, J., Horton, D., Craig, M., Gries, P.,
Evaluating an Inverted CS1, Proceedings of the 45th ACM
Technical Symposium on Computer Science Education, 307312, 2014.
[2] Enfield, J., Looking at the Impact of the Flipped
Classroom Model of Instruction on Undergraduate
Multimedia Students at CSUN, TechTrends, 57 (3), 14-27,
2013.
[3] Gannod, G. C., Burge, J. E., Helmick M. T., Using the
inverted classroom to teach software engineering,
Proceedings of the 30th International Conference on
Software Engineering, 770-786, 2008.
[4] Gehringer, E. F. & Peddycord III, B. W., The InvertedLecture Model: A Case Study in Computer Architecture,
Proceedings of the 44th ACM Technical Symposium on
Computer Science Education, 489-494, 2013.
[5] Hughes, H., Introduction to Flipping the College
Classroom, In World Conference on Educational Multimedia,
Hypermedia and Telecommunications, 2012 (1), 2434-2438,
2012
[6] Lage, M. J., Platt, G. J., Treglia, M., Inverting the
Classroom: A Gateway to Creating an Inclusive Learning
Environment, Journal of Economic Education, 31 (1), 30-43,
2000.
[7] Lockwood, K., Esselstein, R., The Inverted Classroom
and the CS Curriculum, Proceedings of the 44th ACM
Technical Symposium on Computer Science Education, 489494, 2013.