Learning How Pupils Make
Sense of Algebra Word
Problems: Tool
Development
• Robert E. Floden & Algebra Teaching Study
(ATS) Team
• Michigan State University & UC-Berkeley
• This material is based upon work supported by the
National Science Foundation under Collaborative Grant
Nos. NSF collaborative grant DRL-0909815 & DRL090985. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of
the authors and do not necessarily reflect the views of the
National Science Foundation
Support from NSF
• MSU: Robert Floden, Rachel Ayieko, Sihua (Adrienne)
Hu, Jerilynn Lepak Jamie Wernet
• Berkeley: Alan H. Schoenfeld, Evie Baldinger, Danielle
Champney, Fady El Chidiac, Denny Gillingham, HeeJeong Kim, Jerilynn Lepak, Nicole Louie, Sarah Nix,
Daniel Reinholz, Alyse Schneider, Mallika Scott, Kim
Seashore, Niral Shah, and alumni
ATS Team
•
•
•
•
•
Take situations described in words
Build mental models
Represent using algebra symbolism
Operate on symbols
Check results
Classroom learning to make
sense of algebra word
problems
• Link to CCSSM:
• Standards for Mathematical Practice
• Domains
• Expressions & Equations
• Functions
• Mathematics researchers & educators
• Recommendations for instructional practices
• Little empirical work, especially at scale
Why important?
Research on the
Linkage
?
Classroom
Practices
Student
Learning
• Complexities of teaching and learning
• Conceptualize key dimensions of student
learning
• Measuring learning along these
dimensions
• Conceptualizing key dimensions of
classroom practice
• Recording those dimensions
• Keeping mathematics prominent
Challenges for research
•
Robust algebraic understanding
•
•
•
•
•
Interpret text; understand context
Identify salient quantities & relationships
Algebraic representations of relationships
Execute calculations and procedures
Explain and justify
Conceptualizing focus for
student learning
• Draw inferences about competence from
performance on complex tasks
• MARS tasks (built from Balanced Assessment)
• MARS scoring is global, with no detail about
dimensions for robust understanding
• Modest attention to asking for explanations
Measuring student
learning
Illustrative excerpt
• Selection of tasks, largely from MARS
• Cognitive interviews to compare yield from
written response to yield from interview
• Rubrics generate
• Overall score
• Score for each dimension of robust understanding
(robustness criteria or RC)
• Multiple 45-minute forms
Iterative development
of scoring rubric
• Validity studies
• Agreement with holistic scoring
• Agreement of correctness score with MARS
• Comparisons with MARS population
• Reliability studies
• Compare to “expert”
• Random assignment to scorers
• Almost all inter-rater reliability >.75
• Will be higher for classroom means
Reliability and validity
Sample Rubric Hexagons – Part 2
Task (points)
Associated
RCs
Find 10th term in pattern –
perimeter of a row of 10 hexagons
(1)
-
Explain how you know your
answer is correct (1)
RC 5
Possible Strategies
Associated
RCs
Generates representation –
table, graph, or equation
3a, 4a
Generates representation –
diagram of 10 tiles
3a, 4b
Generates pairs of numbers
(e.g. shows recursive
thinking not in a table)
4a
Articulates (in words)
relationship between size
number and number of
people
2b
Draws on representation
3b
Draws on context
(arrangement of hexagons)
4b
• Robustness Criteria As A
Framework To Capture Students’
Algebraic Understanding Through
Contextual Algebraic Tasks
• Sihua Hu, Rachel Ayieko, Jerilynn
Lepak, Jamie Wernet
Scoring rubrics
poster session at PME-NA
Developing a
classroom observation
scheme
• CLASS (UVA)
• IQA (Pittsburgh)
• Plato (Stanford)
• MQI (Michigan)
Review existing schemes
• IQA comes closest:
• Cognitive demand
• Accountable talk
• One thing missing is attention to classroom
dimensions that capture tight plausible link to
developing robust understanding of algebra
What is missing?
• Few publicly available recordings of entire
lessons (about to change with Gates MET)
• ATS data collection Michigan and California
• Evolving agreement on promising dimensions
• Teaching for Robust Understanding of
Mathematics (TRU MATH).
Begin development by
watching algebra
classes
• Mathematical Focus, Coherence &
Accuracy
• Cognitive Demand
• Access
• Agency: Authority & Accountability
• Uses of assessment
Broad dimensions
Level
Mathematical focus,
coherence, and accuracy
1
Classroom activities are
purely rote, OR
disconnected or unfocused,
OR consequential mistakes
are left unaddressed.
2
3
The mathematics discussed
is is relatively clear and
correct, BUT mathematical
justification (ties to
conceptual underpinnings)
is lacking.
The mathematics discussed
is is relatively clear and
correct, AND the
mathematics is well
justified (tied to conceptual
underpinnings).
Mathematical focus,
coherence, and accuracy
Level
1
2
3
Agency: Authority and
Accountability
The teacher initiates
conversations; students’
speech turns are cursory
and effectively constrained
by what the teacher says or
does.
Students have a chance to
say or explain things, but
"the student proposes, the
teacher disposes": in class
discussions, student ideas
are not explored or built
upon.
Students put forth and
defend their ideas, and
teacher ascribes ownership
for students’ ideas in
exposition, AND/OR
students respond to and
build on each others' ideas.
Agency: Authority and
Accountability
•
Robust algebraic understanding
•
•
•
•
•
Interpret text; understand context
Identify salient quantities & relationships
Algebraic representations of relationships
Execute calculations and procedures
Explain and justify
Specific to robust
understanding
RC 3 Rubrics from scheme
Level 1
Generating
representations
•
and 3b
RC 3a of 3a
relationships
between
quantities
Level 2
Level 3
Purposefully
Level 2 and
generated with
attention to why
Generated by way
explicit attention to the representation
of practice
the relationship is a good choice for
between variables the given situation
Representations are Global features of
Level 2 and
Interpreting and
interpreted locally representations are
connections among
making
or in part. No
explicated to
RC 3b connections
multiple
connections
highlight the
representations are
between
between multiple covariation between
explored
representations
representations.
quantities
• Follow teacher, including during group work
• Divide lesson into episodes, with new episode when:
• Move to different mode of instruction (e.g., whole class to
group work)
• Move to new mathematical topic
• When five minutes have gone by since beginning of episode
• Code five general dimensions for every episode
• Code for robust understanding when activity warrants
When and what to
record
• Teaching for Robust Understanding
• Hee-jeong Kim, Kimberly Seashore,
Daniel Reinholz
Scheme poster session at PME-NA
Illustrative preliminary
analyses
• Data are episodes, length of time in
episode, rubric score for episode for each
dimension
• Student learning related to amount of time
spent in activity
• Many possible ways to aggregate, e.g.,
• Average rubric score, weighted by time
• Time spent at particular level of rubric
Aggregating Dimension
Scores
• Data are episodes, length of time in
episode, whether RC-activity happens
during episode, rubric score for RC when it
happens
• Student learning related to amount of time
spent in activity
• Many possible ways to aggregate, e.g.,
• Total time in episodes with RC activity
• Time spent at particular level of RC
rubric
Aggregating RCs Scores
Weighted Average for Dimensions 1-5
3.00
2.80
2.60
2.40
2.20
IM
2.00
CG
A
1.80
AAA
1.60
UA
1.40
1.20
1.00
Tch 1
Obs A
Tch 1
Obs B
Tch 1
Obs C
Thc 2
Obs A
Thc 2
Obs B
Tch 2
Obs C
Tch 3
Obs A
Tch 3
Obs B
Tch 3
Obs C
Tch 4
Obs A
Tch 4
Obs B
4 Teachers-3 Lessons
Tch 4
Obs C
Between-teacher
difference
4 Teachers-3 Lessons
Between-teacher
difference
Average gain in score (%)
Student Gains in Target
Classrooms
• Observation scheme
• Can capture differences among teachers
in general dimensions
• Still working on criteria for recording
events tied to robust understanding
• Student assessment
• Reliable scoring
• Challenges with student motivation
Initial thoughts
• Transitioning From Executing
Procedures To Robust
Understanding Of Algebra
• Rachel Ayieko and Jamie Wernet, with
Sihua Hu and Daniel Reinholz
Paper session at PME-NA
• Working version of rubric for scoring student work, with
all scoring completed for last year’s data
• Close to working version of observation scoring system,
with initial scoring for 3 observations in each of 4 classes
• Exploring approaches to analysis
• Many conference presentations; one publication
• Seeking funding to validate on Gates MET videos
• Exploring use for capturing classroom processes in Shell
Center formative assessment lessons
• Projected paper making comparisons to other observation
schemes
Where we are
• ATS Web site: http://ats.berkeley.edu
• NSF REESE projects:
https://arc.uchicago.edu
For more information