Developing a Computer Science
Concept Inventory for Introductory Programming
Ricardo Caceffo
Steve Wolfman
Institute of Computing
State University of Campinas
Campinas, SP, Brasil
Computer Science
University of British Columbia
Vancouver, BC, Canada
wolf@cs.ubc.ca
rec@ic.unicamp.br
Kellogg S. Booth
Rodolfo Azevedo
Computer Science
University of British Columbia
Vancouver, BC, Canada
Institute of Computing
State University of Campinas
Campinas, SP, Brasil
ksbooth@cs.ubc.ca
rodolfo@ic.unicamp.br
ABSTRACT
In this paper, we discuss our experience developing a CI
for introductory programming courses that cover topics related to basic programming skills, such as variables, conditional commands, loops and iterations, procedures and functions, basic sorting algorithms, vectors and matrix representations, and elementary structures. These topics are far from
exhaustively studied in the CI literature. Our study is part
of a multi-institutional project to develop and validate CIs
for introductory programming courses that can be used with
a peer instruction approach.
A Concept Inventory (CI) is a set of multiple choice questions used to reveal student’s misconceptions related to some
topic. Each available choice (besides the correct choice) is
a distractor that is carefully developed to address a specific misunderstanding, a student wrong thought. In computer science introductory programming courses, the development of CIs is still beginning, with many topics requiring
further study and analysis. We identify, through analysis of
open-ended exams and instructor interviews, introductory
programming course misconceptions related to function parameter use and scope, variables, recursion, iteration, structures, pointers and boolean expressions. We categorize these
misconceptions and define high-quality distractors founded
in words used by students in their responses to exam questions. We discuss the difficulty of assessing introductory
programming misconceptions independent of the syntax of
a language and we present a detailed discussion of two pilot CIs related to parameters: an open-ended question (to
help identify new misunderstandings) and a multiple choice
question with suggested distractors that we identified.
2.
Keywords
concept inventory, data structures, misconceptions
1.
RELATED WORK
Herman et al. [7] identify a CI as a standardized test that
has been statistically validated. Unlike a typical classroom
exam, a CI is not comprehensive. It only covers critical concepts for a topic. Each concept is tested multiple times to
ensure the CI’s validity and reliability [7]. A CI can be used
by an instructor to adjust material to students’ needs. The
idea of a CI was originally applied to fundamental physics:
the Force Concept Inventory (FCI) [8] was created to identify students’ common misunderstandings about Newton’s
Laws of Force.
CIs can also be used to measure the impact of educational
methodologies, such as comparing traditional and interactive engagement educational methods [5]. This is achieved
using pre- and post-tests at the beginning and end of the
term to determine the learning gain of each student through
the course, taking into account any initial knowledge students may have had when the course started.
Some studies reported in the literature [3, 10] developed
CIs based on the flowchart development process proposed
by Almstrum et al. [1]. The first step in developing a CI is
identification of fundamental concepts through synthesis of
various sources (experts, textbooks, papers, etc.). This often
follows a pre-defined path, passing through the development
of open-ended questions, pilot multiple-choice questions, a
student-based investigation, a trial period, and a statistical
analysis, leading to final deployment of the CI. Because CI
development is costly, novel approaches are under development, such as the open-source approach of Porter et al. [12]
and the CS1 exam developed by Tew & Guzdial that uses
pseudo-code questions [14].
INTRODUCTION
A Concept Inventory (CI) is a multiple choice questionnaire used to reveal students’ misconceptions related to a
subject or topic. For each question the answer choices, aside
from the right one, is a distractor designed to address a specific misunderstanding—a wrong yet common way of thinking among students.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than
ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
SIGCSE ’16, March 02-05, 2016, Memphis, TN, USA
c 2016 ACM. ISBN 978-1-4503-3685-7/16/03. . . $15.00
DOI: http://dx.doi.org/10.1145/2839509.2844559
364
A number of initiatives in Computer Science have created,
validated, and applied CIs. Although different methodologies were used, Computer Science CIs can be categorized
into two groups: (1) misconception identification, related
primarily to topics and themes that emerge from students’
misconceptions, and (2) where after identifying misconceptions a pilot CI is developed and validated.
Misconceptions in computer science have been identified
and CI questions proposed for binary search trees (2 questions) [3, 10], digital logic (19 questions) [7], hash tables (1
question) [10], and operating systems (10 questions) [16].
Misconceptions but no questions have been identified for
Boolean logic [6], heaps [3], loops and iteration [9], memory
models and allocation [4, 9], object concepts [9], parameter
scope [4], and recursion [4].
Another approach, proposed by Sorva [13], allows learners
to interact and cognitively engage with a Visual Program
Simulation (VPS). Sorva mapped from the literature 162
misconceptions and categorized them as General, VarAssign
(variables, assignment and expression evaluation), Control
(flow of control during execution), Calls (subprogram invocations and parameter passing), Recursion, Refs (references
and pointers) and OOP (Object-Oriented Programming).
3.
Students were to identify the local and global variables and
describe the expected outputs from printf statements. The
second question asked students to write a C program to input an integer and print all of its divisors. The third question asked students to write a C program to input a year and
print whether or not it was a leap year. The fourth question asked students to write a C program to verify whether
an input n is Pythagorean (n is Pythagorean iff there are
integers a and b such that a2 + b2 = n).
The first question in Exam 2 presented a C function that
correctly implemented Insertion Sort and received as a parameter an int array to be ordered. Students were asked to
create a Date structure composed of year, month, and day
fields, and then change the function to sort an array of Date
structures. The second question asked students to write a
C function to receive an int matrix and verify whether it
followed some pre-defined rules (e.g., the first and last rows
should contain the number 0). The third question asked
students to write a C function to receive an array and two
int pointer parameters and return via the first pointer the
average value of the array and via the second pointer an
element closest to the average value. The fourth question
asked students to write a recursive C function to calculate
the floor of the logarithm base 2 of an int.
Based on exam papers students submitted, we identified
misconceptions and collected them into initial categories that
grouped similar items. We eliminated misconceptions that
fell outside of the scope of our investigation.
METHODOLOGY
We looked at an introductory algorithm and computer
programming course at the State University of Campinas
(Brasil), a mandatory course for all engineering students.
More than 500 students enrol each semester and attend four
50-minute lectures (theoretical classes) and two 50-minute
labs (practical classes) every week. Each week students are
given a practical problem related to theoretical concepts seen
in lecture. They then develop a C programming language
program to solve the problem. This usually requires an additional four out-of-class hours each week. We developed a
CI for the course following an approach similar to Almstrum
et al. [1], which is depicted in Figure 1.
3.2
Instructor Interviews
We conducted semi-structured interviews [11] with 5 course
instructors. We asked them about the main misconceptions
they believed students in the course had and we asked them
to comment on our exam analysis findings. They were also
asked to identify and separate higher-level critical thinking
misconceptions from lower-level syntax misconceptions and
explain how they might apply our CI in their course.
Based on these interviews, we refined misconception categories to reflect the main topics necessary for correct understanding in an introductory programming course. We believe a comprehensive CI for an introductory programming
course must have questions from each of these categories.
Figure 1: CI development methodology, based on
Almstrum et al. [1]
3.3
Development of Pilot Items
We developed open-ended and multiple-choice questions
covering the categories of misconceptions identified in earlier steps, with distractors for each multiple-choice question
drawn from examples in students’ exam papers. These were
used in pilot studies to test the validity of the CI.
Key steps included identification of student misconceptions through analysis of exam papers and instructor interviews, and development of open-ended and multiple-choice
questions for CI items that reflect the misconceptions.
3.1
4.
MISCONCEPTIONS
We identified seven misconception categories. Each category has one or more misconceptions for which we found
evidence in the exams or in our interviews with instructors. Categories represent similar types of misconceptions
and thus offer opportunities to identify linkages between
misconceptions by providing multiple distractors from the
category in a single CI item. For each category we describe
below the type of misconceptions that characterize it and we
give examples drawn from our analysis of exam papers and
our interviews with instructors. We provide the percentage
and number of instances we found of each misconception.
Exam Analysis
We analysed two course exams, each having four questions. The exams were taken individually by 66 and 60 students in the same term, at the end of the second and fourth
months, respectively. We classified student’s errors in exams
according to misunderstandings that might explain incorrect
answers.
The first question in Exam 1 presented a C program with
two functions each called from inside a loop within main().
365
Categories are described in decreasing order of their prevalence.
We also identified an eighth category, which arose in the
interviews, regarding differences between conceptual misunderstandings and misunderstandings about the specific syntax of a construct in a particular programming language.
We discuss that after the seven that arose from students’
exams.
4.1
fore be inappropriate for a CI. We plan to test this in a
multiple choice format and will discard the item if it elicits
few incorrect responses. If it is popular, we will follow up
with think-aloud interviews to assess whether students really
choose the distractor for the reasons we heard in instructor
interviews.
Distractor (c) was identified in both exams. Some students replaced the caller-provided value for a parameter with
a value drawn from some other source. For example, in
Exam 2, some students began a function that received a parameter that was a matrix of integers by creating a loop to
read all matrix elements with scanf("%d", &x). In interviews, some instructors identified cases in which students
started functions by initializing parameter variables with
hard-coded numbers, losing the original values that had been
passed.
Function Parameter Use and Scope
This was by far the largest source of misconceptions, identified in both exams and instructor interviews. Students do
not understand function calls and parameters passing. We
identified three misconceptions in this category.
Call by reference vs. call by value semantics (44%
– 29/66 in Exam 1 and 5% – 3/60 in Exam 2). The
association between actual parameters (arguments) and formal parameters depends on the particular mechanism used
by a language. C uses call-by-value exclusively for all parameters, yet on both exams students thought if they changed
the content of a formal parameter inside a called function,
the actual parameter variable located outside in the caller
function would also change.
4.2
Variables, Identifiers, and Scope
On Exam 1 students did not understand declarations of
variables or rules for assignment and scope in a program.
This category excludes parameters, as explained in the previous section.
Out of scope assignment (33% – 22/66). Variables
declared inside functions were not considered local to the
function. Students classified variables from one function as
being part of another function. Local variables were used as
if they were global.
Parameters as local variables (17% – 11/66). On
Exam 1 we identified cases in which students did not consider function parameters to be local variables. This misconception is a distinct category because it is specific to the
scope of function parameters.
Function names (6% – 4/66). Students classified function names in function prototypes as global variables (they
are global, but they are not variables).
Initialization of formal parameters (12% – 7/60).
On Exam 2, students didn’t appear to understand how parameters work, specifically, how they are initialized. Students apparently did not realize that function calls initialize
parameters so they sometimes gave parameters specific values either by explicit assignment or from an external source
at the beginning of function execution, rather than using the
caller’s supplied values.
Candidate Distractors: Distractors include (a) using
non-variables as variables, (b) using local variables as global
variables, (c) using global variables as local variables, and
(d) having multiple variables with the same name but declared in different scopes (e.g., global and local variables
with the same name).
In the first question of Exam 1, students were asked to
identify the local and global variables inside a program.
Some identified non-variables, such as function names in prototype declarations, as variables (distractor (a)), some identified local variables as global (distractor (b)), while others
identified global variables as local variables (distractor (c)).
In the same question, students analysed a program that had
a loop within the main function that used an int variable i
as iterator. The loop had a call to a function that accessed
an int global variable also named i. Some students got confused and thought the value of i inside the function was the
value of the local variable i inside main, instead of the value
of the global variable i (distractor (d)).
Candidate Distractors: Possible distractors cover situations where (a) parameters are used as if passed by reference, (b) parameters are accessed outside their scope, and
(c) the original parameter value is changed by an external
source such as scanf().
The most instances of errors similar to distractor (a) were
in the first question of Exam 1, in which students had to
printf a local variable after calling a function that receives
the variable as parameter and changes its value. Another
instance of distractor (a) was in the third question of Exam
2, in which students were to implement a function based on
the following prototype:
4.3
void f(double v[], int n, double *a, double* b)
Recursion
On Exam 2 students had three misconceptions about the
application of recursion in problem-solving.
Students should calculate the average value of the elements of v, putting its value into the memory address pointed
by a. Some students instead declared a local variable x, set
it to be the average value, and then set a = &x, perhaps
expecting that some variable in the caller function corresponding to a would change.
Distractor (b) was proposed by instructors in interviews
but was never observed in student responses. It may there-
Bounded recursion (15% – 9/60). The expression for
a return value had an embedded recursive call with a parameter that was larger than the parameter to the current
invocation, leading to an inifite loop because the level of recursion increases indefinitely.
366
Stopping condition (12% – 7/60). Appropriate checks
to terminate recursion when a base case is encountered were
not provided.
as for (int i = 1; i < n; i++) but added inside the loop
another increment i++ of the iterator i (distractor (c)).
Self-reference (7% – 4/60). Functions required to be
recursive did not call themselves.
On Exam 2 students did not understand how to declare
and use C-style structs.
Candidate Distractors: Possible distractors are: (a) a
recursive function without a call to itself, (b) a recursive
call that grows its parameter and thus leads to an infinite
recursive loop, (c) lack of a stopping condition, and (d) use
of an improper stopping condition.
Distractors examples were identified in the fourth question of Exam 2, in which students were asked to implement a
recursive function int logRec (int n) that calculates the
floor of the logarithm base 2 of n. Some students wrote
most of the function code correctly, but didn’t insert a recursive call to the function itself (distractor (a)); other students inserted the correct stopping condition (n == 1) but
the recursive call was made with an increased n (e.g., logRec
(n+1)) leading to an infinite loop (distractor (b)); other students didn’t insert the stopping condition (distractor (c));
and some students inserted an incorrect stopping condition
(e.g., while n == 1), which represents an instance of distractor (d)).
Accessing fields (27% – 16/60). Students tried to use
the struct as an atomic unit (e.g., to compare it to a number
or to another struct), without accessing any of the struct’s
inner fields.
4.4
4.5
Structures
Multi-key sorting (17% – 10/60). Given an array of
Date structs, students created three variants of the insertion
sort function that had been provided and sorted the array
first by year, then by month, and finally by day. They instead should have made a single multi-key sort function.
Candidate Distractors: A possible distractor relates
to the use of the whole struct when only a field is needed.
For example, on the second question of Exam 2, students
should compare two Date structs (a and b) with day, month,
and year fields. Instead of comparing the struct through
their fields (e.g., a.day > b.day, a.month > b.month, and
so on) the distractor would compare the entire struct using
the statement a > b.
Iteration
4.6
On Exam 1 students seemed not sure when iteration should
be used or how the number of loop iterations is managed.
Pointers
On Exam 2 students did not understand the concepts of
memory addresses and pointers, nor why and how pointers
should be used.
Basic loop behavior (14% – 9/66). Students used a
loop to perform some calculation but had a call to printf()
for the result value that appeared inside the loop, not outside the loop.
Returning values through pointers (3% – 2/60).
Students created a local variable containing the value to be
returned by a function and assigned its memory address to
the pointer parameter, rather than dereferencing the pointer
parameter to assign the value to be returned to a memory
location external to the function.
Logic of looping count (14% – 9/66). Students did
not pass all required values (from 1 to n) to an auxiliary
function. Instead, they only called the function once, without any looping structure.
Assign values to pointers (12% – 7/60). Students
assigned to a pointer variable a specific value instead of a
memory address.
Candidate Distractors: Possible distractors are (a) premature final result calculation or reporting before the loop
is over, (b) solving a problem that requires iteration without
having a loop, (c) improper initialization of the loop counter,
and (d) improper updating of the loop counter.
An instance for distractor (a) was identified in the fourth
question of Exam 1, in which students were supposed to
verify if an integer n is Pythagorean. The correct answer
involves a loop to check all possibile pairs of numbers. Some
students inserted inside the loop a printf stating whether n
was or was not Pythagorean before testing all possible number combinations. In the same question, some students implemented an auxiliary boolean function that receives three
parameters a, b, and n and verified if the condition for n
to be Pythagorean was satisfied for the parameters a and
b but some of those students only called the function once,
without making a loop to check all possibilities for a and b
(distractor (b)).
An instance for distractor (c) was identified in the first
question of Exam 1, in which students were asked to print all
numbers that are divisors of some integer n. Some students
initialized a for loop with i = n instead of i = 1. In the
same question, some students correctly coded the iteration
Candidate Distractors: Possible distractors are: (a)
assigning a value to an address, (b) assigning an address to
a value, and (c) assigning to a pointer an invalid address.
For example, in the third question of Exam 2, students
should make a function that receives two double* variables
a and b and changes the values pointed to by a and b to
some calculated double x. Some students assigned x to an
address, for example a = x (distractor (a)); some assigned
the address of x to the value pointed to by a using the statement *a = &x (distractor (b)); and others assigned to the
pointer the address of x using a = &x, which would point
to an invalid address after the function returns (distractor
(c)). The expected answer was *a = x.
4.7
Boolean Expressions
Students did not understand boolean expressions’ meanings or their evaluation.
Truth tables (12% – 8/66). Incorrect use of the ||
and && operators.
367
choice pilot items based on categories of misunderstandings.
In open-ended questions students can write the answers in
their own words, allowing discovery of new misconceptions.
Multiple-choice questions have the advantage of being easier
to analyse than are open-ended questions, but they require
creation of distractor options to address each identified misconception.
We explored the open-ended questions approach during
the exam analysis described in Section 3.1. We opted to
create both new open-ended questions (to identify new misconceptions) and multiple-choice pilot items. Due to space
limitations we present here only two questions, which relate to the topic Function Parameter Use and Scope
as examples to illustrate our approach. Question 1 (Q1)
is an open-ended question, to help identify new misconceptions related to the topic. Question 2 (Q2) is a multiplechoice question that has distractors for the misunderstandings identified already in our study.
Candidate Distractors: Possible distractors cover (a)
translation from English sentences to incorrect Boolean expressions, which don’t have the expected precedence order
or semantics; and (b) attempts to create a Boolean expression without boolean operators (e.g., without ‘and’ (&&) and
‘or’ (||)).
An instance for distractor (a) was identified in the third
question of Exam 1, in which students were asked to verify if
a number n represents a leap year. Leap years are multiples
of 400 or they are multiples of 4 that are not multiples
of 100. Some students were not able to correctly translate
this sentence to a Boolean expression. For example, some
used only && or || operators without including parenthesis
to delimit the precedence order. Other students tried to
solve the problem through an if sequence, not using any
Boolean operators at all. Although not technically wrong,
this workaround does not contemplate the purpose of the
exercise (distractor (b)).
4.8
Q1. Function usage. Write a program in C that asks
the user to enter two int numbers, a and b. Then, if a is
equal or greater than b, calculate and print a-b. Otherwise,
calculate and print b-a. You must write one (and only one)
auxiliary (helper) function that performs the subtraction.
Syntax vs. Conceptual Understanding
In our interviews, instructors raised concerns about distinguishing between learning/conceptual misconceptions and
highly language-specific syntactic misconceptions. For example, could one or more CI items distinguish between a
student who does not grasp the concept of pointer addressing and one who understands the concept but not the C
progamming language syntax for pointer addressing? The
consensus of the instructors who were interviewed was that
the nature of the subject (programming) makes it hard to
elaborate conceptual questions entirely independent of the
particular programming language being used. This conclusion is similar to findings in the literature [9].
Another concern raised by instructors focused on the typical use of CIs as pre- and post-tests. Unlike the Force
Concept Inventory [8], where college students have everyday experience of basic Newtonian force concepts and may
even have studied them, many students’ first contact with
most programming concepts happens in college introductory
programming courses. A CI that assumes knowledge of programming language syntax at the start of the term would not
be effective; it could even frighten students. To address this
issue, we propose an approach similar to that discussed by
Porter et al. [12], where the pre-course CI consists of conceptual questions, not related to programming language syntax,
and the post-course CI has those same questions, but with
additional specific programming language questions added.
This approach is related to Commonsense Computing, an
attempt to get at deep computing concepts in ways accessible to people with no computing background [15].
This leads to the development, for the same misunderstanding, of two types of CI questions: conceptual or analogy questions, accessible to all students at any point in
the course, and technical or language-dependent questions,
where knowledge of the chosen programming language syntax is required. This approach supports the measurement of
learning gains; it is still possible to compare the initial subset of questions that are in both the pre- and post-course
tests. Developing such questions remains a goal.
5.
Discussion: This open-ended question was developed to
help find new misconceptions related to function parameters’ use and scope. The fact that the student must write
just one auxiliary function verifies if the student understands
how parameter passing works. This question can also help
identify misconceptions related to variable assignment and
scope and boolean expression evaluation.
Q2: Scope of variables. The following code includes a
function that adds five to a number. This is called on x in
the main function.
int addFiveToNumber (int n) {
int c = 0;
// Insert a Line Here
return c;
}
int main ( ) {
int x = 0;
x = addFiveToNumber(x);
}
The correct line to be inserted is:
(a) scanf
(b) n = n
(c) c = n
(d) c = x
("%d", n);
+ 5;
+ 5;
+ 5;
Discussion: Aside from the right answer (option (c)), each
other option matches a specific distractor presented in Section 4.1: option (a) relates to the change of the original
parameter value by an external source; option (b) relates to
the misunderstanding that parameters are passed by reference, and option (d) relates to the misunderstandings that
a variable can be accessed outside its scope. Our interest in
piloting this question is how attractive the distractors are.
PILOT TEST QUESTIONS
As Almstrum et al. propose [1], the next step in this
kind of research is the creation of open-ended or multiple-
368
For example, the scanf() distractor does not include any
addition of 5. Will students avoid it for that reason? As we
further develop the question, we will tailor it to try to elicit
the actual misconceptions that students hold, based on our
experience with the pilot exam.
6.
[4] K. Goldman, P. Gross, C. Heeren, G. L. Herman,
L. Kaczmarczyk, M. C. Loui, and C. Zilles. Setting
the scope of concept inventories for introductory
computing subjects. Trans. Comput. Educ.,
10(2):5:1–5:29, June 2010.
[5] R. Hake. Interactive-engagement versus traditional
methods: A six-thousand-student survey of mechanics
test data for introductory physics courses. Am. J.
Phys., 66(1):64–74, January 1998.
[6] G. L. Herman, M. C. Loui, L. Kaczmarczyk, and
C. Zilles. Describing the what and why of students’
difficulties in Boolean logic. Trans. Comput. Educ.,
12(1):3:1–3:28, Mar. 2012.
[7] G. L. Herman, M. C. Loui, and C. Zilles. Creating the
digital logic concept inventory. In Proceedings of the
41st ACM Technical Symposium on Computer Science
Education, SIGCSE ’10, pages 102–106, New York,
NY, USA, 2010. ACM.
[8] D. Hestenes, M. Wells, and G. Swackhamer. Force
Concept Inventory. Phys. Teach., 30(3):141–158,
March 1992.
[9] L. C. Kaczmarczyk, E. R. Petrick, J. P. East, and
G. L. Herman. Identifying student misconceptions of
programming. In Proceedings of the 41st ACM
Technical Symposium on Computer Science Education,
SIGCSE ’10, pages 107–111, New York, NY, USA,
2010. ACM.
[10] K. Karpierz and S. A. Wolfman. Misconceptions and
concept inventory questions for binary search trees and
hash tables. In Proceedings of the 45th ACM Technical
Symposium on Computer Science Education, SIGCSE
’14, pages 109–114, New York, NY, USA, 2014. ACM.
[11] J. Lazar, J. H. Feng, and H. Hochheiser. Research
Methods in Human-Computer Interaction. Wiley
Publishing, 2010.
[12] L. Porter, C. Taylor, and K. C. Webb. Leveraging
open source principles for flexible concept inventory
development. In Proceedings of the 2014 Conference
on Innovation #38; Technology in Computer Science
Education, ITiCSE ’14, pages 243–248, New York,
NY, USA, 2014. ACM.
[13] J. Sorva. Visual Program Simulation in Introductory
Programming Education. PhD thesis, Aalto University,
Finland, 2012.
[14] A. E. Tew and M. Guzdial. The FCS1: A language
independent assessment of CS1 knowledge. pages
111–116, 2011.
[15] T. VanDeGrift, D. Bouvier, T.-Y. Chen,
G. Lewandowski, R. McCartney, and B. Simon.
Commonsense computing (episode 6): Logic is harder
than pie. pages 76–85, 2010.
[16] K. C. Webb and C. Taylor. Developing a pre- and
post-course concept inventory to gauge operating
systems learning. In Proceedings of the 45th ACM
Technical Symposium on Computer Science Education,
SIGCSE ’14, pages 103–108, New York, NY, USA,
2014. ACM.
CONCLUSIONS AND FUTURE WORK
We have begun to identify misconceptions appropriate to
introductory computing courses using the C programming
language that cover concepts related to basic programming
skills, such as variables, conditional commands, loops, functions, pointers, and structs. Through a careful exam analysis, we first identified the main misunderstandings associated with each topic. Then, aided by instructor interviews,
we categorized the misconceptions, extracting the distractors. We are now applying these data to generate a pilot
CI, composed of 21 open-ended and multiple-choice questions, classified either as conceptual or analogy questions or
as programming language-dependent questions.
The CI is available at the http://edu.ic.unicamp.br/
caceffo website.
In future work, we plan to test our pilot CI to statistically validate it and use think-aloud interviews to identify
new, not-yet-addressed misconceptions. We plan to apply
the conceptual/analogy questions as a pre-course test, and
to also include them in the post-course test as a means to
measure learning.
Another approach that we plan to take is an adaptation
of the Peer Instruction methodology [2], called CSPI (Computer Science Peer Instruction). Currently in development,
CSPI addresses Computer Science-specific needs, for example, supporting the use of classroom clickers to test specific
misunderstandings of programming concepts such as a program’s execution flow control. We plan to apply the CI
that we develop to measure the CSPI’s educational impact
in CS1 courses. Future work will continue the multicultural
approach in the work to date, with the CI application tested
in universities from different countries, with different native
languages, and different student backgrounds.
7.
ACKNOWLEDGMENTS
This research was supported in part by the São Paulo Research Foundation (FAPESP) under grants #2014/07502-4
and #2015/08668-6 and by the Natural Science and Engineering Research Council of Canada (NSERC) under grant
RGPIN 116412-11.
8.
REFERENCES
[1] V. L. Almstrum, P. B. Henderson, V. Harvey,
C. Heeren, W. Marion, C. Riedesel, L.-K. Soh, and
A. E. Tew. Concept inventories in computer science
for the topic discrete mathematics. SIGCSE Bull.,
38(4):132–145, June 2006.
[2] C. H. Crouch and E. Mazur. Peer instruction: Ten
years of experience and results. American Journal of
Physics, 69:970–977, 2001.
[3] H. Danielsiek, W. Paul, and J. Vahrenhold. Detecting
and understanding students’ misconceptions related to
algorithms and data structures. In Proceedings of the
43rd ACM Technical Symposium on Computer Science
Education, SIGCSE ’12, pages 21–26, New York, NY,
USA, 2012. ACM.
369