Assessment in mathematics
education: The interface between the
multiplicative conceptual field and the
Rasch measurement model
Caroline Long, CEA, University of Pretoria,
Caroline.long@up.ac.za
Presentation outline
• Challenges for mathematics education
• Theoretical tools
• Theory of conceptual fields (Vergnaud, 1983)
•
Threshold concepts (Meyer & Land, 2005)
• Zone of proximal development (Vygotsky, 1962, 1978)
• Research study
• Rasch measurement theory
• Reflections on the outcomes
• Educational implications
• Application of the Rasch model in mathematics education
settings
• Further developments
Challenges for mathematics teaching
•
The transition from working with whole number to rational numbers
(and real numbers), is noted as a critical mathematical development in
the transition years, Grades 6 to 10 (Usiskin, 2005), which in large
part determines the mathematical future for learners.
•
Fractions, ratio and related topics like percent, are taught procedurally
without regard for their complex interconnections and their
applications in everyday situations (Kieren, 1976; Parker &
Leinhardt, 1995; Lamon, 2007).
•
It is the assimilation and accommodation of the component
subconstructs comprising rational number that are required for
synthesis in the predicative form (Kieren, 1976; Vergnaud, 1983;
Lamon, 2007).
•
The danger is that “tricks” are introduced to avoid real engagement
with the concept (Davis, 2010).
Theory of conceptual fields (Vergnaud, 1983 - )
• Mathematics is developed through encountering concepts
embedded in problem contexts
• No one-to-one mapping from mathematical concept to problem
situation. In fact all such mappings are many-to-many
• Concepts are never learned singly but in relation to other
concepts
• The multiplicative conceptual field include
•
•
•
at base multiplication and division, but also
the problem situations which are modelled by multiplication and division
and
the various representations that may be required.
• The purpose of mathematics education is to transform extant
implicit and local conceptions of individual learners
encountered in single problem situations into generalised
mathematics concepts and theorems that may be applied to a
class of related problems – from operational to predicative form
Threshold concepts (Meyer & Land, 2005)
•
An historical perspective
•
•
•
increasingly abstract systems, radical conceptual shifts (Dantzig, 2007)
parallel conceptual reorganisation in the classroom (Sfard, 1991)
resonates with the notion of “epistemological obstacles” (Brousseau, 1983)
•
A threshold concept is a critical concept which provides the
gateway (to higher mathematics), and which by corollary inhibits
mathematical progress where learners have not gained mastery.
•
These conceptual gateways may be
•
•
•
•
•
transformative – “occasioning a significant shift in perception of a subject”
“troublesome” knowledge - defines “critical moments of irreversible
conceptual transformation”
irreversible – “unlikely to be forgotten”, “unlearned only through
considerable effort”, and
integrative – “exposing the previously hidden interrelatedness of concepts”
The danger of substitution by naïve concepts which confine progress
Zone of proximal development (Vygotsky,1962)

Theory of conceptual fields builds on the work of Piaget and Vygotsky
•
“(t)he discrepancy between a child’s actual mental age and the level
he reaches in solving problems with assistance indicates the zone of
his proximal development” (Vygotsky,1962, p. 102).
•
Current interpretation - a conceptual and cognitive space comprising
the distance or extent from current proficiency to a higher level, over
which learning with assistance from teachers or peers may occur .

There may be hierarchies of concepts that build on one another and
that some concepts may be within the learner’s current “zone of
proximal development”. More abstract concepts may require prior
mastery or at least partial mastery. For example an understanding of
ratio may precede an understanding of time distance and speed.
Assessment instruments
 The contextual features discussed lead us to assert that educational
experiences in mathematics are ordered somewhat hierarchically
and aligned with learners current proficiency levels.
 How do we identify for individual learners, or groups of learners a
suitable current conceptual space, an educational locale (or zone of
proximal development) which is neither too tedious because the
associated problem solutions are obvious, nor too difficult for the
learner and hence frustrating or demotivating?
 Can assessment instruments be designed that make explicit a
conceptual and cognitive pathway that provides insights for the
teacher, in the preparation of learning sequences and subsequent
engagement with individual learners? And in this process, can
instruments help identify threshold concepts?
Research design
 Instrument construction, 36 items (18 MC and 12 polytomous
items)
• TIMSS 2003 released items, adapted slightly in some cases
• Matrix design (12 common items and a further 24 distributed
across 4 groups)
 Learners at two schools, across three grades, 7, 8 and 9, 16
classes, and 330 learners
 Reasonably well-functioning schools, each covering a range of
South African demographics
 Follow-up interviews on 4 items of graded difficulty with 6
learner groups (21 learners in all, located at specified
proficiency levels as determined from a Rasch analysis).
Person-Item Map
PERSONS [locations=estimated proficiency]
7.0
6.0
Persons and items
on the same scale,
according to
probabilistic
estimates.
Selected ratio, rate
and proportion items
are described on the
right.
Levels* aligned with
logits are
demarcated by
bands from (- ,-2)
[-2, -1), [-1,0), [0, 1),
[1,2), [3, + )
5.0
4.0
3.0
2.0
1.0
0.0
-1.0
-2.0
-3.0
o
= 2 Persons
ITEMS [locations=estimated difficulty]
o |
|
| 34
|
|
|
|
|
30. Find the average speed
| 26
given distance, 160 km, and
|
time, 2.5 hours.
|
|
o |
o |
|
|
| 27
|
o |
|
10. 2.4 litres for 30 hours,
o |
how many litres for 100
o |
hours?
oo | 35
| 31 30
ooo |
o |
oo |
oo |
o | 28
5. Identifies proportional
ooooo | 7
ooooo | 32
share of amount 45 000
oooooooooo | 29 10
divided in the ratio 2:3:4.
ooooooo |
ooooo | 18 19
ooooooooooooo | 9 8 15
oooooooooooo | 33 16
oooooooooooo | 14 25 5
20. Selects fraction
oooooooooo |
representing part to whole
ooooooooooooooooooo | 3 21
ratio.
ooooooooooo | 23
oooooo | 4 24
ooooooooo | 20 6 12 11
ooooooooo |
ooooooooo | 22
oooo |
ooooo | 2 17
1. Given a proportional
o | 13 1
problem, able to apply
ooooo |
multiplicative reasoning,
|
o |
1:3:: 9:x.
|
A note on terminology
 “Levels”
• overused, a place holder,
• familiar (Griffin, 2006; Pisa 2009; TIMSS)
• denotes a defined position on a unidimensional line
 “Locale”
• an area, a domain, or range on the 1D line
• a difficulty locale (for a subset of problem situations)
• a proficiency locale (for a subset of learners currently
exhibiting a similar proficiency)
 Criterion zone (Wilson, 2005)
Item analysis by level and analytic category
Mathematical
situation
Notation
Range
Mathematical structure
Proportional
sharing
Natural
language,
whole
numbers
Less
than 30
20. Selects
fraction
representing
part to whole
ratio.
Part-whole
comparison
Inclusive ratio
Fraction
notation
 30
16 of total
(16 + 14)
Level 6
[1,L 2)
Level 4
[0,1)
Level 3 [-1, 0)
1. Given a
Collecting
proportional
bottles
problem, apply
multiplicative
reasoning,
1: 3 :: 9: ?
Level 1
[- ∞, -2)
Context
Level 2
[-2,-1)
Description
Birthday
M1
1
9
16
16  14
5. 45 000 zeds Family
shared in
sharing
unequal
money
proportions, of
2, 3 and 4.
Proportional
sharing
10. 2.4 litres to Capacity30 hours, how time
many for 100
hours?
Proportion
Multiplicative
comparison
30. Average
speed given
distance and
time.
Rate
calculation
Car rally,
Distance,
time,
speed
Natural
numbers in
thousands,
fraction
and ratio
notation
Decimal
notation
Decimal
notation

45000
(1000)
 100

M1
(time)
first half
total
16
30
2:3:4
4 of 45 000
M1
4
9
9
f (x) =
2 .4
30
 160
160 km
=
2 .5 h
64km/h
d
t
 s
Response process
M2
3
x
Identify
multiplicative
relationship
Selects part-whole
M2
(children) relationship
16
30
M2
x
45
Identifies
proportional
relationship
Identifies
M1
M2
(capacity) (hours) proportional
2,4
30
relationship and
x
100
multiplicative
operator
Identifies
M1
M2
(hours)
(km) proportional
2.5
160
relationship and
1
x
calculates unit rate
Distractor analysis
Item 1 (n=330)
Quartile Low Q ML Q MH Q High Q
Item analysis
Selected items (Item 1, Item
20, Item 5, Item 10, Item 30)
at graded levels of difficulty,
Mean %
58
84
93
98
Simple ratio
Item 20 (n=83)
Quartile Low Q ML Q MH Q High Q
Mean %
28
70
81
86
Part-whole comparison
levels of proficiency: means
of 4 quartile groups (LQ,
MLQ, MHQ, HQ).
The key mathematical
concepts for success on the
item noted below.
Item 5 (n=330)
Quartile
Mean %
24
35
55
85
Multiplicative relationship, unequal
proportions
Item 10 (n=330)
Quartile
Mean %
Multiple choice distractor
plots: percentage of quartile
group selecting each false
choice
Low Q ML Q MH Q High Q
Low Q ML Q MH Q High Q
8
19
28
65
Finding the unit rate
Item 30 (n =330)
Quartile Low Q ML Q MH Q High Q
Mean %
0
0
5
41
Understanding the relationship of distance
speed and time
MCF concepts embedded
in the problem situations
at each level.
Associated errors by level
• Part-whole of discrete and
continuous quantities
• Both fraction and ratio meaning
of fraction notation
Level 2 [-2-1)
Errors
• Confusion between fraction
measure and ratio meaning
and with fraction notation
LQ MLQ MHQ
(-1.9) (-1.1)
(0)
HQ
(1.1)
53%
88%
93%
94%
37%
61%
79%
91%
16%
35%
57%
78%
14%
22%
28%
58%
0%
3%
9%
29%
• Multiplicative relationship
between sets of ratios
• Fraction equivalence
• Part-part and part-whole ratios
• Percent concept and notation
• Connect probability with fraction
measure
•
•
•
•
Natural number confusion
Just add the percent sign %
Language difficulty
Ignoring part of the problem
• Covariant relationships
• Comparative relationships
• Rational number, operator
subconstruct
• Identification of ratio
Level 3 [-1,0)
Summary
information
Level 1[-3,2)
MCF concepts
• Multiplicative comparison
(operator construct)
• Confusion with operator
construct
• Confusion of additive and
multiplicative relationship
• Ignoring part of the problem
• Percentage increase (operator
subconstruct)
• Applying ratio operator construct
to find the sample
• Fraction measures, addition
(subtraction), multiplication
(division)
• Ratio and rate concepts
• Multiplication (division) of
decimals
• Consider only the
numerator
• Additive reasoning
• Lack of fluency with
multiplication and division
• Confusion with terminology
• Probability and statistics
concepts, “sample”, “random”
• Rate and ratio
Level 5 to 7
( > 1)
Mean percentages
correct for items by level
and by quartile
Level 4 [0, 1)
• Multiplicative comparison
• Percent, identifying referents
• Covariant relationships
• Reasoning with unknowns
• 2 step problems
• Confusion with percent
language and referents
Interviews: Item level by learner proficiency
Item 5: Three brothers Thabo, Samuel and Dan, receive a gift of 45 000 zeds from their
father. The money is shared between the brothers in proportion to the number of children
each one has. Thabo has two children, Samuel has three children, and Dan has 4 children.
How many zeds does Dan get?
Carola, (0.18 logits),
Interview data
Shiluba, (0.26 logits)
I didn’t really completely understand but I got a
part answer. My logic behind it … I divided 3 into
the R45 000 … And this gave me R15 000. … I
think I then timesed by two over one which gave
me 30, which I then divided by three which then
gave me R10 000.
Linda, (0.12 logits)
I realise I did it wrong … really bad … I worked
out all of them together. All the children. And I
divided them by the zeds. Then I got 5 000. Then
I think you would divide the 5 000 by 4.
…
I did that wrong, I don’t know why.
Kate,(0.12 logits)
I said 45 000 divide by 3 and I got 9 000. And
then I don’t know what I did and I got 5 000
Interviews with the
middle high proficiency
group, on Item 5.
Written responses on
the left, explanations of
thinking about the
problem on the right.
I divided all of them
4 500 divide by 2 …
…
For Sam, for Thabo, and I did the same for
Samuel (divide by 3) and Dan (divide by 4).
…
I don’t know why I did that (laughs, and group
laughs with her).
I never worked out how much Dan alone gets.
If I had divided 5 000 by 4, I would have got the
answer.
You mean multiply by 4.
I have no idea (and laughs)
Interview
transcript
Speaker
Transcript
Interviewer
OK, so has anybody got a picture of what is happening here?
Shiluba
We were supposed to add up the children and divide it into 45 000. And then we
would get 5 000.
And what does that 5 000 mean. 5 000 for … Dan?
No, 5 000 for each of the children
For each of the children …
(Murmurs of realisation from the group)
You get 15 000.
(The group corrects her.)
You get 20 000.
Interviewer
OK. Did you get that?
(Confirmation and chuckling from the group)
So where was the problem? Was it in the reading?
Shiluba
In the “How many zeds does Dan get?” Because I didn’t really, … I kind of shut
off … Which is that part?
The middle part?
Ja, “The money is shared between the brothers in proportion to the number of
children each one has”.
OK,
So I disregarded the children.
Item 5, middle
high quartile
group transcript
Discussion
occurred after
individual
explanations of
reasoning about
the problem.
Common errors are noted
in brackets,
Low
(L1, 2)
Fluency with addition
and subtraction
(Mistakes, use of addition
where multiplication
more efficient)
(Use of addition where
multiplication more
efficient)
Fluency with
multiplication and
division, proportional
sharing
(Confusing multiplication
and division in
proportional shares)
Some fluency with
multiplication and
division, sharing
(Lack of fluency with
multiplication and
division;
Abandon sense making
Applying incorrect
algorithm)
Some fluency with
multiplication and
division, sharing
(Confusing multiplication
and division;
Incorrect conception of
proportional sharing)
Some fluency with
multiplication and
division
(Misreading of the
problem)
Conversions, operations
with decimal fractions
Conversions of
measurement units
Conversions from
fraction to percent
Multiplying decimal
fractions
Fluency of operations
with decimal fractions
Conversions from
percent to decimals
Percent change percent increase
Percent - fraction type
Converting ratio
relationship to a
percentage
Finding percent
increase (finding percent
decrease)
Percent, fraction type
(Mixing objects, adding
percents and quantities)
(Confusing percent with
amount, rather than ratio.
Difficulty changing
referent when converting
ratio to percent)
Using ratio
understanding to find
proportions, finding
unit rate
Reasoning with ratio
and rates (Lack of
fluency, multiplication
and division)
Identifying variables
and relationships
Identifying scalar and
function operators
Identifying variables
and relationships
(Confuses relationships
incorrect use of
algorithms)
(Incorrect identification
of relationships, working
back from the answer)
(Incorrect identification
of relationships; working
backwards from the
answer)
Fluency with
algorithms
(Applying algorithms in
conceptually clumsy
ways, using inappropriate
algorithms
Misuse of equals sign)
(Incorrect use of
algorithms, Mistakes with
cancelling, when applying
algorithms)
(Rote use of algorithm,
sometimes correct)
(Rote use of algorithms)
Multiply
and
divide
Fluency with addition
and subtraction
Fractions,
decimals,
operation
On the vertical axis, the key
concepts identified are
listed from lesser difficulty
to greater difficulty.
Middle low
(L3)
Percent,
concept,
operations
On the horizontal axis the
proficiency levels are
arranged from high to low.
Middle high (L4)
Using ratio
and
rate concepts
Composite information
from summaries of item
analyses at levels of
difficulty and summaries of
interview data at levels of
proficiency.
High
( L 5,6,7)
Identify
Algorithms, symbolic
variables and
notation
relationships
Composite summary
Add and
subtract
Levels
Description of concepts and skills in bold, (errors enclosed in brackets)
Reflections: theory of conceptual fields and
Rasch measurement
• The insights gleaned from such a study have application in their local
context, for example for the population targeted in the study, Grades
7, 8 and 9 in South African schools.
– While some learners have yet to achieve fluency with
multiplication and division, others need extension at the higher
end of the spectrum of multiplicative structures, for example,
rate and double proportion problems
– The selection of proximate items not yet correctly mastered by
learners is easily executed given the outcomes of the Rasch
analysis, in conjunction with the theoretical analysis of the
multiplicative conceptual field.
– The method advanced in this study follows that advocated
Wilson (2005) for the most part, though further cycles of
refinement may be implemented for any context.
The interactive relationship of theory and
measurement
• This development and validation may elicit joint orderings of
sets of items within a conceptual field, and respondents, the
distances between which have a meaningful interpretation
and permit the specification of a ZPD.
• We may infer from these relationships, positing educationally
pertinent interventions.
• We claim that, given theoretical analysis, we may infer a
degree of hierarchical ordering that, with further cycles
replicated in these grades, may prove to be a helpful
instrument in the planning and support of teaching.
• Provided that the instrument has been carefully constructed to
include potential threshold concepts, these thresholds may be
hypothesised and perhaps identified.
Concluding remarks
 The theory of conceptual fields raises some questions about
current articulations of the intended curriculum
 From relationships of item difficulty and learner ability,
educationally pertinent interventions can be targeted at groups
of similar learners, as a means of reaching all learners
efficiently – notion of ZPD
 The identification of threshold concepts - somewhat more
complicated
• From an historical perspective, the topics which caused particular
consternation, for example the critical concepts that led to the
construction of new number systems, the zero, the integer, the
rational number etc., may be the starting point.
• The theoretical insights (see for example, Dantzig, 2007) into the
topics which appear particularly troublesome and which require
attention
 Critical for threshold concepts to be hypothesised and then
included in assessment research
Further developments: A model of assessment
(Bennett & Gitomer, 2009)
Accountability
component
Professional
development
component
Formative
component
Accessible and powerful assessment
 Accountability
• A process of teacher involvement at the various phases of
the item construction
• Accountability becomes intrinsic to the teacher
 Professional development component
• Deep domain specific knowledge
• Expert input on identified problem areas
 Formative assessment
• A classroom component where teachers develop,
experiment and reflect on the suitability of current
assessment in relation to the knowledge domain
• Professional learning communities (PLCs)
Critical questions for future research
 How may the theoretical notions of
conceptual fields (threshold concepts and
zones of proximal development) inform
educational assessment and measurement?
 And reciprocally, how may measurement
theory highlight and extend insights into
conceptual fields, threshold concepts and
zones of proximal development as they
manifest in the progressive development of
learning?
Thank you for your attention
Caroline.long@up.ac.za