Inlernalional Journal of Matheinalical EdiicaUon in
Science and Technology, Vol. 36, N o . 7, 2005, 701 712
f f ^ Taylor & Francis
V ^
Undergraduate students' performance and confidence in procedural
and conceptual mathematics
JOHANN ENGELBRECHT*. ANSIE HARDING
and MARIETJIE POTGIETER
University of Pretoria, South Africa
{Received 29 April 2005)
The general perception is that high .school teaching of mathematics in South
Africa tends to be fairly procedural and thai siudenls that enter university are
better equipped to deal with procedural problems rather ihan conceptual. This
study compares the conceptual and procedural skills of first-year calculus
students in life sciences. Also investigated is students' confidence in handhng
conceptual and procedural problems. The study seems to indicate that these
students do nol perform better in procedural problems ihan in concepuial
problems. They are more confident of their ability to handle conceptual problems
than to handle procedural problems. Furlhermore the study seems to indicate
that students do nol have more misconceptions about conceptual mathematics
than about procedural issues.
I. Introduction
Mathematics pedagogy based on Vygotskian theory approaches mathematics as a
conceptual system rather than a collection of di.sciete procedures. Vygotsky [1] noted
that the possibilities of genuine education depend both on the knowledge and
e.Kperience already existing within the student (level of development) as well as on
the student's potential to learn.
While this eonceptual base is important, the communication system for algebra
and other high school mathematies topics can be considered as a procedural
approach to teaching mathematics. In South Africa, high school teaching of
tnathematics tends to be fairly proeedural. Although most students arriving at
university have well developed manipulation skills in mathematics, few of these
students have really been exposed to deeper eonceptual thinking. University teachers
often complain that tirst year students have little understanding of any of the
basic concepts ol" precalculus and even the better students are only better in a
procedural way of thinking. With this background, we investigate the procedural
and conceptual skills of first year students, also attempting to tneasure
their confidence in handling procedural and conceptual approaches in first-year
calculus.
"Corresponding author. Email: Jengelbrd:' up.ac.za
InlcriHilioniil Jaiinuil nf Mullumaiicut Ediiiiilioii in Svifnic and Tfchnoliifiy
ISSN 11020 719X prim ISSN 1464 52M online . 2005 Taylor & Francis
hllp: . www.iiincir.ai.Ilk, jciurniils
DOI: l().U)S(I.Oli:()7.W()5OO27l 107
702
J. Engelhrccht et iii.
2. Conceptual and procedural knowledge in mathematics and
eontidence of response
Conceptual and procedural knowledge in mathematics is a topic addressed by many
researchers. Most authors agree in principle but there are subtle ditferenees in
interpretation that we attempt to explicate in the following literature exposilion.
In its 'Learning standard for mathematics' the New York State Education
Department [2] distinguishes between conceptual understanding and procedural
fluency:
Conceptual understanding consists of those relationships constructed internally and connected to already existing ideas. It involves the understanding of
mathematical ideas and procedures and includes the knowledge of basic
arithmetic facts. Students use conceptual understanding of mathematics when
they identify and apply principles, know and apply facts and definitions, and
compare and contrast related concepts. Procedural fluency is the skill in
carrying out procedures flexibly, accurately, efficiently, and appropriately. It
includes, but is not limited to. algorithms (the step-by-step routines needed to
perform arithmetic operations).
Hicbcrt and Lefevre [3] describe procedural knowledge as 'composed of the
formal language, or symbol representation system ... [and] the algorithms, or rules,
for completing mathematical tasks'. They continue to assert that procedural
knowledge is meaningful only if it is linked to a conceptual base.
2.1. Teaching procedures and concepts
Teaching for procedural knowledge means teaching definitions, symbols, and
isolated skills in an expository manner without first focusing on building deep,
connected meaning to support those concepts [4]. Teaching for conceptual
understanding, on the other hand, begins with posing problems that require students
to reason flexibly. Through the solution process, students make connections to what
they already know, thus allowing them to extend their prior knowledge and transfer
it to new situations [5].
Brown ct al. [6] are of the opinion that conceptual knowledge means that
students must make sense of mathematics. The difierence between teaching for
conceptual knowledge and the traditional way of ending chapters with
application problems is fundamental. For conceptual learning the applications
are often at the beginning of the chapter and the mathematics is drawn from
them. They are not merely a place for applying previously mastered skills, lime
permitting, as is the case in a more traditional setting. So while they agree that
both procedural and conceptual knowledge are important, the key issue, as
they sec it, is in the manner and order in which procedures and concepts are
taught. 'Teaching first for conceptual knowledge leads to the acquisition of
procedural knowledge, but the converse is not true" [6].
2.2. Relationship between procedural and conceptual thinking
The main reason for the reform movement in the teaching of calculus courses (in
particular) was the emphasis on procedural knowledge and it has been claimed [7]
Proeedurat and conceptual performance and confidence
that studenls' conceptual knowledge will necessarily increase their procedural
proficiency.
In Anderson's model of learning [8], learning begins with actions on existing
conceptual knowledge. The student begins to internalize the procedures involved,
leaving aside the conceptual knowledge from which the procedures arose. So the
conceptual knowledge changes into procedural knowledge. The process of acquiring
procedural knowledge depends upon existing conceptual knowledge and the knowledge gained by the repeated use of procedures [6]. For Piagel Ihis process develops
further. After the student has gained proficiency in procedural knowledge, a process
of reflection begins and as a result new conceptual knowledge develops [6]. For
Piaget conceptual knowledge and procedural knowledge are both integral parts of a
single cognitive schema; they are not separate. In the Piaget model procedural
efficiency is a requirement for metacognition and conceptual thought [9], For
Vygotsky [I] on the other hand, algebraic thought begins with conscious reflection
upon existing unconscious conceptual knowledge.
A study by Baker and Czarnocha [9] found that conceptual thought is
independent of an individual's ability to apply his or her procedural knowledge,
supporting Vygotsky's view that development can proceed through reflection upon
existing conceptual knowledge independently of the reflection due to repeated
actions.
A relevant question is whether it is possible to have conceptual knowledge/
understanding about something without having procedural knowledge. Baker
et al. [10] address this issue; "In [the] "traditional" curriculum, concept development
is viewed as arising from computational proficiency with relevant procedures'.
Calculus courses often begin with a brief review of definitions and then focus on
computational modeling of procedural knowledge grounded in algebra. Aspinwell
and Miller [II] agree with this view: "students regard computation as the essential
outcome of calculus and thus end their study of calculus with little conceptual
understanding'. The 'dynamic action view' model [7] is an example of a model
based on procedural knowledge. Here learning takes place by applying procedural
knowledge to an existing conceptual foundation in which case increasing proficiency
in procedural knowledge assists in expanding conceptual knowledge.
Gray and Tall [12, 13] and Tall et al. [14] take the analysis to a next level. They
introduce the idea of a procepi as the symbolisation of an object that arises from
processes carried out on other objects. Such procepts can then be viewed in two
distinct but related ways, as a process or as an object. A procept is considered as a
cognitive construct, in which the symbol can switch from a focus on process to
compute or manipulate, to a concept that may be thought of as an entity that can be
manipulated. Tall et at. [14] believe that procepts are at the root of human ability to
manipulate mathematical ideas in arithmetic, algebra and other theories involving
manipulable symbols. They allow the biological brain lo switch effortlessly from
doing a process to thinking about a concept in a minimal way.
Tall et al. [14] consider the word procedure as a specific .sequence of steps carried
out a step at a time, while the term process is used in a more general sense to include
any number of procedures with 'the same effect'.
Those who are procedurally oriented are limited to a particular procedure, with
attention focused on the steps themselves, whilst those who sec symbolism as
process or concept have a more efficient use of cognitive processing [14].
703
704
J. Etigelimrhi ct al.
Recognizing the same distinction of components. Dubinsky el al. [15. 16]
introduce the ideu of processes being eucapsuhiu-d as objects. They embed this
encapsulation in what they call the APOS model for the construction of conceptual
mathematical knowledge, describing how Actions become Processes that can be
viewed as Objects, as part of Schetmis. The components of this model: Action,
Process, Object and Schema represent an increasing level of learning. An action is a
change that an individual makes in a mathematical context requiring precise
instructions to perform. A process takes place when the individual begins to have
control over the concept. An nhjcci is constructed from a process when the individual
becomes aware of the entire concept and understands that actions or processes can
act on the concept. A schema is when objects and processes from more than one area
can be combined in more than one way.
Chappell and Killpatrick [17] conducted a study involving 305 undergraduate
calculus students in which two learning environments were created, one in which a
procedural approach was followed and the other in which a conceptual approach
was followed. The students exposed to the concept-based learning environment
seored significantly higher than the students in the procedure-based environment on
assessment that measures conceptual understanding as well as procedural skills.
These results provide post-secondary evidence that a concept-based instructional
programme can effectively foster the development of student understanding without
sacrificing skills proficiency.
2.3. Confidence of response
The Confidence of Response Index (CRI) has its origin in the soeial sciences, where it
is used particularly in surveys and where a respondent is requested to provide the
degree oi certainty he has in his own ability to select and utilise well-established
knowledge, concepts and laws to arrive at an answer [18]. In an academic examination
environment a student is asked to provide an indication of confidence of response
along with each answer set. This indication is usually based on some scale, say (0-5).
where 0 implies a total guess and 5 implies complete confidence. Irrespective of
whether the answer is correct or not. a low conlidence indicates a guess which, in turn,
implies a lack of knowledge. However., if the confidence is high and the answer wrong
it points to a misplaced confidence in his knowledge on the subject matter, either
misjudging his own ability or a sign of the existence of misconceptions. Hasan
et al. [19] use the confidence of response, in conjunction with the correctness or not of
a response can thus be used to distinguish between a lack of knowledge (wrong answer
and low confidence) and a misconception (wrong answer and high confidence). This
may not always be the case; students could just be over confident or in procedural
problems students with high confidence may make numerical errors.
3. Research question
In view of the discussion above on procedural and conceptual knowledge in
mathematics, we use the following working definitions for the two concepts for
the purpose of this study:
Procedural knowledge is the ability to physically solve a problem through the
manipulation of mathematical skills, such as procedures, rules, formulae,
algorithms and symbols used in mathematics.
Procedural and conceptual peiforniance and confidence
705
Comepiual knowledge is the ability to show understanding of tnathematical
concepts by being able to interpret and apply them correctly to a variety of
situations as well as the ability to translate these concepts between verbal
statements and their equivalent mathematical expressions. It is a connected
network in which linking relationships is as prominent as the separate bits of
infonnation.
The objective of the study is to determine whether there is any relation between
students" conceptual and/or procedural understanding in mathematics and whether
there is any relation between their confidence levels when handling procedural and
conceptual problems. Furthermore we want to investigate the relationships between
students" confidence and their actual performance in procedural and conceptual
mathematical problems.
4. The experiment
In order to compare the coneeptual and procedural abilities of students, we used a
group of first-year students in life sciences. These students all do an introductory
course in applied caleulus in the mathematics department. The number of students in
the sample group is 235.
The test consists of ten multiple-choice items of which five are eonsidered to be
predominantly procedural and W\c predominantly conceptual (by the authors). We
realized that virtually ail test items required both conceptual and procedural
thinking to he solved satisfactorily. The items were mixed in the question paper.
For construct validity, the test was thoroughly and independently scrutinized by
colleagues in the tnathematics department, all involved in the same course, to get an
unbiased view ofthe the percentage of procedural knowledge [kp) and the percentage
of conceptual knowledge (A,.) needed to complete each item successfully according to
the working definitions ofthe constructs mentioned earlier. The panel was also asked
to indicate (in their opinion), the level of dilficulty on a procedural (d^,) and
coneeptual level (c/.) of each item in the test. We used the averages of the panel
members" opinions after a discussion to confirm or dispute disparities in individual
opinions. We realise, in agreement with Anderson [1]. that the concepts procedural
and conceptual are not absolute conceptual problems can become procedural if
students are exposed to the same type of problem repeatedly. Despite this, an
impressive cohesion of opinions was experienced.
An example of an item considered to be (more) conceptual by the panel, is:
Which ofthe following graphs/(.v) satisfies both the given conditions
(i) ,/'(.v) > 0 on (-0C. 0) and /"(.v)<() elsewhere
(ii) f"{.\) > 0 on (-ex:, - 1 ) and/"(.v)<0 elsewhere
7
E. Ntine ol'ihe above.
706
J. Engelhrecht el al.
According to the panel, for this item: k,, = {i.\'&. A, =0.82, rf^= 1.25 and c/,. = 2.5.
An example of an item considered to be more procedural by the panel, is:
For the function /t.v) = x \ what is the equation of the tangent line at .Y= 1?
A. r = A- + 2
B. r = 3.V - 2
C. y ^ X
D.y = 3.V + 2
E. None of the above
For this item: A'/, = 0.89. k, =0.11, (/^,- 1 and (/,.— 1 (according to the panel).
On completion of each of the items, the student had to indicate how confident
he/she was about the answer to the question, choosing between completely certain,
fairly certain, not certain and a total guess.
For each student we calculate a procedural performance index (PPI) and a
eoneeptual performance index (CPI). The PPI is calculated using the formula
where a= 1 if the question is answered correctly and a = 0 if not. The sum is taken
over the number of relevant items. The conceptual performance index is similarly
calculated by
Both these indices are expressed as percentages.
For the certainty of response or confidence, we use similar formulas. We
introduce a procedural confidence index (PCI) and a conceptual confidence index
(CCI) for each student, calculated as follows:
(where the value of /i being 3. 2. 1 or 0, indicating complete certainty, moderate
certainty, uncertainty or a total guess respectively). Similarly
By using these formulas, we neutralise the influence of the level of difliculty of a
question. Again these indices are expressed as percentages.
We make the following comparisons:
(1) We compare the performance of each student in the conceptual items (CPI)
with his her performance in the procedural items (PPI). For this comparison,
we disregard all students who indicated that their answers were a total guess
for any item, leaving us with 130 students for this comparison.
Procedural and conceptual performance and confidence
1^1
(2) We compare the procedural and conceptual confidence indices (PCI and
CCI) for all students in the sample.
(3) We compare the procedural confidence index (PCI) with the procedural
performance index (PPI) and the conceptual confidence index (CCI) with the
conceptual performance index (CPI) for all students.
5. Research results and discussion
In all three compari.sons we construct a scatter plot for the data. We divide the
graphing area into four quadrants (numbered anti-clockwise beginning with the top
right quadrant) splitting the vertical and horizontal axes at the corresponding
average value for all students for ihc rclevanl index. We calculate the Pearson
correlation coefficient and we compare the average values for the entire group in
comparisons I and 2, verifying the comparison with a statistical /-test.
5.1. Comparison I: Conceptual performanee (CPI) vs procedural
performance (PPI)
To get a true reflection of students" performance, all students who indicated at any
item that their answer is a 'total guess' are disregarded in this comparison. This
means that the sample size shrinks to 130 students. The numerical results for
comparison I can be seen in table I.
Figure I represents the scatter plot of the students' performances on the
conceptual and procedural aspects of test items. The interesting quadrants are the
second quadrant containing students that perform well conceptually but poorly
procedurally, and the fourth quadrant with students that do well procedurally but
not so well conceptually. The first quadrant contains students performing well both
conceptually and procedurally and the third quadrant students thai perform poorly
in both aspects.
The average CPI for this sample is 60.6 and the average PPI is 57.4. This
result came as a surprise. In spite of the fact that the averages do not differ
significantly (/J = 0 . 2 7 ) . there is a leaning towards conceptual rather than procedural performance. The correlation between conceptual and procedural performance indices is 0.29. which, although significant for the sample size, is lower than
expected. It is also interesting to note that more students perform conceptually
Table 1. CPI vs PPI.
PPI
Sample size
Average for group
Pearson correlation
/)-value (/-test)
% students in quadrant
% students in quadrant
% studenis in quadrant
% students in quadrant
CPI
130
57.4
1
2
3
4
60.6
0.29
0.27
35
19
30
16
J. Engelbrecht ct ul.
708
100
75*
fe 50.
25
0
25
50
100
75
PPI
Figure 1,
Scatter plot of sltidctits" CPI vs PPI.
Table 2,
CCI vs PCI,
CCI
Sample size
Average for group
Pearson correlation
/7-value (/-test)
% students in quadrant
% students in quadrant
% students in quadrant
% students in quadrant
235
62,5
I
2
3
4
65.6
0,63
0.006
33
15
37
15
we]l and proceduraUy poor]y (19%) than proccdiira]ly wcl] and concepluaUy
poorly (16%).
5.2. Comparison 2: Conceptual confidence (CCI} vs procedural confidence (PCI)
In this case aU sludents in the satTip]e arc ta]4en into account, a satnple size of 237
students. The statistics for comparison 2 can be seen in table 2.
Figure 2 represents the confidence indices of the students. In this case students in
the second quadrant (15%) have a high conceptual confidence level but a weak
procedural confidence level and the fourth quadrant contains students (15%) that
are confident procedurally but not so well conceptually. These numbers do not differ
signiticantly and are botli fairly low.
Students" confidence on conceptual items (average 65.6%) is significantly
(/J —0.006) higher than their confidence levels for procedural items (average
62.5%). The correlation between their conceptual and procedural confidence indices
is high (0.63). The good correlation is confirmed by the scatter graph.
Procedural and conceptual performance and confidence
709
100
75
8
25
25
Figure 2.
50
PCI
100
75
Scatter plol of students* CCI vs PCI.
Table 3.
PCI vs PPI und CCI vs CPI.
Sample size
Pearson correlation
% sludents in quadriint 1
% sliidcnts in qtiiidr;im 2
% students in quadr;tnl 3
"••^) students in quadr;tnl 4
PCI vs PPI
CCI vs CPI
0.391
30
20
0.380
33
19
32
33
IS
16
5.3. Comparison 3: Confidence rv Performanee (CCI vs CPf and PCI vs PPI).
In these comparisons again all students in the sample are taken into account, giving a
sample size of 237 students. The numerical results for comparison 3 can be seen in
table 3. while the scatter plots for these comparisons arc shown in figures 3 and 4.
Correlations between students" confidence and their actual performance for both
the procedural and conceptual items (both about 0.4) are significant for the sample
size but lower than in any ofthe other two comparisons. Comparing the numbers in
the fourth quadrants (performance higher than confidence), we notice the numbers
of students that perform better than what they expect conceptually than procedurally
arc very similar. Comparing the numbers in the second quadrants (conlidence higher
than performance), we notice that the numbers of studenls thai are overconfident
conceptually and procedurally, are also very similar. Following the classification of
Hasan el al. [19]. this would imply that no more misconceptions exist among students
about conceptual malhemalics than about procedural mathematics.
6. Conclusions
This study seems to indicate that students in this course do not perform better in
procedural problems than in conceptual problems. This finding supports the
710
J. Engelhrecht et al.
•
•
75
•
JI
•
*
:
i
• •
t
t
•
V
<
•
*
•;
•
•
t
•
*
'
I
50;
o
Q.
;
•
;
25 :
*
•
_
•
-
n
0
25
50
75
100
PPI
Figure 3.
Scatter plot ol'students" PCI vs PPI.
100
75
.
O
50
t
I
.
• .
••
25 .
25
Figure 4.
50
75
100
CPI
Scatter plot of students* CCI vs CPI.
evidence of Chappell and Kilpatrick [17] that conceptual understanding does not
come at the cost of sacrificing skills proficiency.
Our study also indicates that students are more confident about their ability to
answer conceptual problems than for procedural problems. The general opinion that
'doing' is easier than "thinking' is disputed.
Are there plausible explanations for this unusual fitiding? The course followed
by students involved in this study, an introductory course in applied calculus,
follows a reform calculus approach. New concepts are introduced in four ways:
verbally, numerically, algebraically and visually. The emphasis is on understanding and interpretation. For example, rather than drilling students in the
Proceditral and conceptual performance and confidence
711
dififerentiation process, a premium is placed on the meaning of a derivative, what it
tells you and how it can be used. This approach differs from how the mainstream
ealculus course is presented. Within the context of a teaching approach that
cultivates conceptual thinking the findings of this study stand to reason. It may
well be the case that the findings of this study do not hold true when a similar sttidy
is conducted with students for whom a dillercnt teaching approach is followed. For
this reason it is important to expand the study to other groups and even to other
disciplines. The study does provide a tool for investigating this issue.
The study also seems to indicate that students do not have more misconceptions
about conceptual issues than about procedural mathematics. This finding is also
somewhat surprising and refutes the general perception that misconceptions are
more likely to appear in conceptual thinking than when working with procedures.
References
]l] Vygotsky. L,, 1986. Thottght attd Langttage (Cambridge. MA: MIT Press),
[2] New York State Education Department. 2005, Learning Statidards for Mathematies.
Retrieved on 28 March 2005 from http:'/www,ernscnysed.gov/ciai/mst/mathstandards/
inlro,html
[3] Hiehert, .1, and Lefevre. P,, I9i^6. In J, Hiebert (Ed.). Cottceptiial atid Procedural
Knowledge: The Case of Mathenuttics (Hillsdale. NJ: Erlhaum),
[4] Skemp. R,, 19S7, The Psyehology of Leavnitig Mathetnaties {H\\\<.&A\C, NJ: Erlbaum),
[5] National Council of Teaehers of Mathematics. 2000, Principles atui Standards for
School Mathematies (Reston, VA: National Couneil of Teachers of Mathematics),
http://www.emse,nysed,gov/ciai/mst,'mathstand;trds/intro,html
[6] Brown. S,. Seidelmann, A, and Ziminermann. G,. 2002, In the trenehes: three teaehers'
perspectives on moving beyond the math wars. Retrieved 25 Mareh 2005 from
http://mathemalicallysane,eom,. analysis, trenches,asp
[7] Haapasalo, L, and Kadijevich. D,, 2000, Two types of mathematical knowledge and (heir
relation. Journal ftir Maihettiatikdidatik. 21(2), 139-157.
[8] Anderson, J,R,, 1995, Cognitive Psychology atid it.s Implicatiotts, 4th edn (W. H. Freeman).
[9] Baker, W. and Czarnocha. B.. 2002, Written meta- cognition and procedural knowledge.
Proeeedings ofthe 2nd International Conference on the Teaching of Mathematies. at the
undergraduate level. University of Crete. July 2002, Hersonissos. Crete.
[10] Baker. W., Czarnoelia, B. and Prabhu V,. 2004. Proeedural and conceptual knowledge in
mathematics. Proceeding.^ of the North Atnerican Chapter of the Internatiotud Group for
the Psychology of Mathetnatics Edtteation Annual Cottference, October 2004. Toronto,
Ontario, Canada.
[11] Aspinweil. L, and Miller. D,, 1997. Students' positive reliance on writing as a process to
learn first semester calculus. Journal of Iti-itiiutional Psychology.. 24, 253-261,
[12] Gray, E,M, and Tall. D.O,. 1991. Duality, ambiguity & flexibility in sticeessful
mathcmatieal thinking. Proceeding.-; of PME XV. Assisi. Italy. 11. 72 79,
[13] Gray, F.M, and Tall. D,O,, 1994. Duality, ambiguity and flexibility: a proceptual view of
simple arithmetic, Jourtnd of Research in Mathematics
Education. 26(2). 115-141,
[14] Tall. D,. Gray. E,, Bin Ali, M,. Crowley. L,, DeMarois. P,. McGowen, M,. Pitta. D,,
Pinto. M.. Thomas. M, and Yusof, Y., 2001. Symbols and the bifurcation between
procedural and conceptual thinking, Canadiati Jmtrttal of Science. Mathetnatics and
Technology Edtteation. 1. 81-104,
[15] Dubinsky! F,. 1991. In D.O. Tall (Fd,). .Advanced Mathetnatical Thitiking (Dordreeht:
Kkiwer). pp, 95-123.
[16] Thotnas. M,O,J. atid Hollon, D,. 2003. In A,J. Bishop, M,A, Clements. C. Keitel.
J, Kilpatrick and F.K.S, Leung (Eds). Second Interttatiotial Ilandhook of .Mathetnatics
Education. Vol, 1 (Dordreeht: Kluwer), pp, 347 390,
712
J. Engelhrecht el al.
117] Chappell. K.K. and Kiilpatrick. K.. 2003. EtVects of concept-based instruction on
students' conceptual understanding and proccdtiral knowledge of calculus. Primus.
Xlll(l). 17 37.
[18] Webb. J.M.. 1994. The eilect of feedback timing on learning facts; the role of response
conlidence. Contemporary Educational Psychology. 19, 251 265.
[19] Hasan. S.. Bagayoko, D. and Kelley. E.L,. 1999. Misconceptions and the Certainty of
Response Index, CRI. Physics Education, 34, 294 299.