Candace Walkington
Assistant Professor of Teaching and Learning
Southern Methodist University
Presentation Overview
 Personalization of Learning
 Theoretical Framework - Interest
 Pilot Work
 Study in Algebra I classrooms
 Summary, Conclusions, Next Steps
Student Motivation in the 21st
Century
 Important issues with student motivation face schools
today (Hidi & Harockwicz, 2000), especially in secondary
mathematics (Mitchell, 1993)
 Interest in mathematics declines over adolescence
generally, and algebra classes specifically (Fredicks & Eccles,
2002; Frenzel, Gotez, Pekrun, & Watt, 2010; McCoy, 2005)
 Algebra I a gatekeeper to higher-level mathematics (Kaput,
2000), with significant implications for equity & access
(Cogan, Schmidt, & Wiley, 2001; Moses & Cobb, 2001)
Student Motivation in the 21st
Century
 Failure rates in Algebra I continue to be high, especially
among low-income students and student of color
(Allensworth, Nomi, Montgomery, & Lee, 2009; McCoy, 2005)
“I think we're growing serfs in our cities,
young people who graduate with eighth grade
education that can't access economic
arrangements to support families. Kids are
falling wholesale through the cracks – or
chasms – dropping out of sight… people say
they do not want to learn. The only ones who
can dispel that notion are the kids
themselves.” ~Bob Moses, Algebra Project
Student Motivation in the 21st
Century
How can learning technologies be utilized to enhance
student motivation and promote achievement in
difficult secondary subjects like Algebra I?
“I think we're growing serfs in our cities,
young people who graduate with eighth grade
education that can't access economic
arrangements to support families. Kids are
falling wholesale through the cracks – or
chasms – dropping out of sight… people say
they do not want to learn. The only ones who
can dispel that notion are the kids
themselves.” ~Bob Moses, Algebra Project
Personalization of Learning
 Learning technologies emerging that personalize
instruction to background, goals, preferences, and prior
knowledge (e.g., Papert, 1980, 1983; Carnegie Learning, 2011)
Personalization of Learning
 Learning technologies emerging that personalize
instruction to background, goals, preferences, and prior
knowledge (e.g., Papert, 1980, 1983; Carnegie Learning, 2011)
 Learners accustomed to customization, interaction, and
control when seeking knowledge outside of school (Collins
& Halverson, 2009)
 National Academy of Engineering named “Advancing
Personalized Learning” Grand Challenge for
engineering in 21st century
Context Personalization
 Matching instructional components with students’
personal interests and experiences (e.g., sports,
gaming, movies, etc.).
 Most research has been conducted in elementary
mathematics, with mixed results
 Domain: Algebra I
You work at a furniture store
and make $10.50 per hour.
How much money will you
make in 5 hours?
You work at a video game
store and make $10.50 per
hour. How much money will
you make in 5 hours?
Theoretical Framework
 Interest: the psychological state of engaging and the
predisposition to re-engage with certain objects, events, or
ideas (Hidi & Renninger, 2006)
 Context personalization may elicit topic interest –
triggered when learners are presented with a specific topic
or theme (Ainley et al., 2002)
 Activating interest can involve:
 Affect: Emotions accompanying engagement with the topic
(Hidi & Renninger, 2006)
 Value: The feelings of importance or worthwhileness the
learner ascribes to a topic (Schiefele, 1991, 2001)
 Knowledge: Learner’s knowledge of the procedures and
discourse related to the topic (Renninger et al., 2002)
Theoretical Framework
 Personalizing instruction may spur affect or stored
value for a topic
 May also trigger relevant stored knowledge about the
topic, that is related to the task at hand:
 Reasoning with familiar quantities in algebra
(Carraher et al., 2006; Chazan, 1999; Lampert, 2001)
 Grounding of abstract ideas in concrete experiences
(Goldstone & Son, 2005) - redundancy with everyday
knowledge supports inferences (Koedinger, Alibali, &
Nathan, 2008)
Theoretical Framework
 Interest has been associated with:
• Attention, persistence, engagement (Schiefele, 1991;
Schiefele & Krapp, 1996; Hidi, 1995, 2001; McDaniel, Waddill, Finstad, &
Bourg, 2000; Renninger & Wozinak, 1985; Ainley Hidi, & Bendorff, 2002;
Ainley, Hillman, & Hidi, 2002; Flowerday, Schraw, & Stevens, 2002)
•
Motivational variables like self-efficacy, self-
regulation, achievement goals (Harackiewicz, Durik, Barron,
Linnenbrink-Garcia, & Tauer, 2008; Hidi & Ainley, 2008; Sansone et al.,
2011)
• Learning (Ainley, Hillman, & Hidi, 2002; Ainley, Hidi, & Bendorff,
2002; Harackiewicz et al., 2008; Hulleman & Harackwicz, 2009; Schiefele
1990; 1991)
Pilot Study
 24 Algebra I students given story problems (y = mx+b) that
were either normal or personalized to their interests
 Modifications based on a pre-interview
“Every time I tweet my numbers go up - I get
more followers… [I have] 156 followers…”
 Personalized problems easier to solve for struggling
students, and harder linear functions
 More informal strategies, less conceptual errors
 Students reported personalized problems as easier, more
related to their lives
 What about long-term learning?
Research Question
Can a context personalization intervention
aimed at matching instruction to topics
students are interested in promote long-term
learning in algebra?
 How does personalization impact performance while the
intervention is in place?
 How does personalization impact performance once the
intervention is removed?
 What is the impact of personalization for students who
struggle with algebra?
Participants
 145 Algebra I students at a Pennsylvania high school
 School used Cognitive Tutor Algebra curriculum
 Adapts hints, feedback, and problem selection
 Story problems & multiple representations
 Unit 6 “Linear Models and Independent Variables”
Method
 Students given open-ended survey about their out-of-
school interests in 9 topic areas (sports, music,
movies, TV, games, computers, art, food, shopping)
 Results of survey used to write problems that were
“personalized” to these 9 different topic interests
 4 variations written for each original problem in Unit 6
Method
Interest
Normal
Problem
(Control
Group)
Food
Sports
Stores
Movies
Problem Text
An experimental liquid (LOT#XLHS-240) is being tested to
determine its behavior under extremely low temperatures. Its
current temperature is -35 degrees Celsius and is slowly being
lowered by two and one-half degrees per hour.
A new soda at McDonald’s is being tested to determine its
behavior under extremely low temperatures. Its current
temperature is -35 degrees Fahrenheit and is slowly being
lowered by two and one-half degrees per hour.
A new sports drink is being tested to determine its behavior
under extremely low temperatures. Its current temperature is 35 degrees Fahrenheit and is slowly being lowered by two and
one-half degrees per hour.
…
…
Method
 Participants randomly assigned to 2 conditions in
Unit 6:
 Control: Receive normal problems for Unit 6
 Experimental: Receive 1 of 4 personalized versions of
same problems for Unit 6, based on interests survey
Method
Analysis
 Tutoring environment tracked different concepts or
knowledge components (KCs):
 Easy: entering a given, identifying units and quantities
 Medium: RU/SU, write expression slope only
 Hard: Write expression with slope and intercept
Performance Effects
 Significant impact for easy (3% difference, p < .001) and
hard (10% difference, p < .001)
 Personalization significantly reduced time spent writing
symbolic expressions (6.93 second reduction, p < .05)
Percent Correct
100%
***
Control
Experimental
80%
60%
***
40%
20%
0%
Easy
Medium
Hard
Performance Effects
Average Accuracy
0.7
Control
Experimental
0.6
0.5
0.4
0.3
0.2
1
2
3
4
5
6
7
8
9 10 11 12 13
Number of Opportunities to Practice
Condition*
Opportunity
interaction
significant, p < .05
Performance Effects
 Students with low performance in algebra identified
Percent Correct
through curriculum progress measures (13 C, 12 E)
 Personalization had a significantly greater impact on
performance for struggling students (24%, p < .05)
60%
50%
40%
30%
20%
10%
0%
*
*
Control
Experimental
Non-Struggling
Student
Struggling Student
(Hard only)
* p < .05
Gaming the System
 Issue with Intelligent Tutoring Systems
 Enter answers quickly and repeatedly
 Click through to “bottom out” hint
 Baker and deCarvalho (2008) developed “gaming
detector” that utilizes log data
 Personalization significantly reduced gaming
behaviors in Unit 6 (p < .05, Cohen’s d = 0.35)
Learning Effects
 Next expression-writing section (Unit 10)
 Stories and equations more complex
 Experimental group still significantly better at
writing expressions in Unit 10 (p < .01)
 6% difference in expression-writing in Unit 10 (40%
control vs. 46% experimental)
 Also maintain learning efficiency gain (6.26 second
reduction; p < .01)
 Learning impact greater (3 times larger) for weaker
students (attrition)
Conclusions
 Interventions designed to elicit interest have the
potential to support learning, even in advanced
domains like algebra
 Adaptive technology environments that personalize
instruction can impact learning of difficult skills
 Need to further explore how interest can be leveraged
in adaptive environments, more authentically
 Develop stronger theory behind learning from
personalization interventions
Future Directions
 Expand intervention to 4 units
 Collect measures at multiple grain sizes of how
personalization interacts with:
 Affective states (boredom, frustration, engagement)
 Triggered and maintained situational interest
 Individual interest and utility value
 Self-efficacy
 Achievement goals
 Metacognitive strategies and “gaming the system”
Future Directions
 More authentic interventions:
 Classroom study of teachers implementing
“personalized” versus “normal” units – richer problem
contexts or mini-projects
 Classroom study where Algebra I students generate
their own “personalized connections” – utility value
interventions
 “Personalized” visual representations (NCCMI)
 Analysis of Mathia data sets (used in over 100 schools
in first year)
Significance
 A recent national survey of Algebra I teachers found that:
 “Working with unmotivated students” as most challenging
aspect of teaching algebra
 Second place: “Making mathematics accessible and
comprehensible to all students.”
(Loveless, Fennel, Williams, Ball, & Banfield, 2008)
 Adaptive learning technologies offer a potentially powerful
means to enhance motivation and achievement for students
who struggle with mathematics
Acknowledgements







Milan Sherman
Anthony Petrosino
Mitchell Nathan
Jim Greeno
Ken Koedinger
Ryan Baker
Vincent Aleven
 The Pittsburgh Science of Learning Center
 Carnegie Learning
Papers
 Walkington, C., & Maull, K. (2011). Exploring the assistance dilemma: The case
of context personalization. In L. Carlson, C. Hölscher, & T. Shipley (Eds.),
Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp.
90-95). Boston, MA: Cognitive Science Society.
 Walkington, C., & Sherman, M. (2012). Using adaptive learning technologies to
personalize instruction: The impact of interest-based scenarios on
performance in algebra. In Proceedings of the 10th International Conference of
the Learning Sciences. Sydney, Australia.
 Walkington, C., Sherman, M., & Petrosino, A. (2012). ‘Playing the game’ of
story problems: Coordinating situation-based reasoning with algebraic
representation. Journal of Mathematical Behavior, 31(2), 174-195.
 Walkington, C., Petrosino, A., & Sherman, M. (in press). Supporting algebraic
reasoning through personalized story scenarios: How situational
understanding mediates performance and strategies. Mathematical Thinking
and Learning.
 Walkington, C. (under review). Using learning technologies to personalize
instruction to student interests: The impact of relevant contexts on
performance and learning outcomes. Invited to special issue of Journal of
Educational Psychology.