Improving a Mathematical
Intelligent Tutoring System
Experiments & Equation Solver Improvements
July 27th, 2012
Jennifer Ferris-Glick & Hee Seung Lee
Mathematical Intelligent
Tutoring Systems
• How do we improve out dated textbook
methods of learning and expand on intelligent
tutoring systems?
Objective of Study
• To investigate the role of verbal explanation
- When does it help learning?
• To Explore the degree to which we can teach
without verbal instruction
- Is verbal direction necessary?
- How can we help students discover/learn underlying
problem structure and induce correct rules for problem
solving without explicit verbal directions?
Linearize task
: solving algebra problems using data-flow diagrams
: allows to study algebra learning anew in college population
5 -
5 *
4 *
+
5 +
29
29
Findings from Linearize study
• Showing intermediate cognitive steps
• Preventing superficial analogy from an example
to problem
• Building appropriate problem representations
• Avoiding excessive floundering
• Giving a chance to reflect on solved problems
Showing Intermediate Cognitive Steps
• Students still have to fill in gaps between steps
when students study a worked example
• Students learned better when intermediate
cognitive steps were clearly revealed than when
students had to determine hidden steps.
 Force students to study step-by-step solutions
Preventing Superficial Analogy
• Given a highly similar example, students tend to draw
a superficial analogy from the example to problem.
– When superficial solution is available, students fail to
abstract deep problem structure.
– When examples look too dissimilar from a target
problem, students sometimes simply ignore or do not
spend enough time to study provided examples.
 Examples should be dissimilar, and structurally
similar to a target problem.
Building Appropriate Problem
Representations
• Revealing hidden structure helped transfer
- Making connections to Algebra activated prior knowledge
and helped understand the problem structure
• Highlighting relevant features (e.g., color coding)
 Highlight features to direct
student’s attention and reveal
hidden problem structure
Restricting Floundering &
Supporting Reflection Activity
• When students show too much floundering, they
may not remember solution paths. In order to
reduce floundering:
– Immediate feedback on wrong attempt
– Providing hints when students show floundering
– Auto-solving with a time limit
• Giving students an opportunity to make sense of
already solved problems
 Allow only a certain number of off-track
activities. When a certain step was solved
automatically, show corresponding verbal
explanations for a while.
General Improvements to Carnegie
Learning Cognitive Tutor
(all conditions)
Showing Intermediate Cognitive Steps
– Force step-by-step guided
example before solving
• Maybe show previously
solved equations
– Hint improvements
• Some changes but not full
review
Preventing superficial analogy
– Present side-by-side example
– Similarity of example
• Randomly selected from a
pool of problems in each
section (not too similar, not
too dissimilar)
Appropriate Problem Representation
– Aligning equations on the equals
sign
– Coloring variable sides of
equation
Restrict Floundering
– Reduce off-path steps
• either allow 0 or 1 off-path
step
Reflection Time
– Provide side-by-side already
worked out problems
Target Material
One-step linear equations
• Solving with addition and subtraction (no type in)
– o w + 6 = -1
• Solving with addition and subtraction (type in)
– o -4 = 1 + z
• Solving with multiplication and division (no type in)
– o 2x = -1
• Solving with multiplication and division (type in)
– o 8w = 5
Target Material
Two-step linear equations
• Solving One-step equations (Type in)
– o Mix of types from unit 5
• Solving with multiplication (no type in)
– o -2 = 5x + 10
• Solving with multiplication (type in)
– o -2 = 5x + 10
• Solving with division (no type in)
– o x/6 + 7 = 8
• Solving with division (type in)
– o x/6 + 7 = 8
• Solving with a variable in the denominator (no type in)
– o 10 = -2/x
• Solving with a variable in the denominator (type in)
– o 2/z = -4
Target Material
Linear equations with similar terms
• Solving two-step equations (type in)
– o Mix of types from unit 6
• Combining like variable terms and a constant with integers (no type
in)
– o 2 – 9x + 6x = 2
• Combining like variable terms and a constant with integers (type in)
– o -6x + 2 + 5x = -6
Combining like variable terms with decimals (no type in)
– o 9 = 9.7x – 2x
• Combining like variable terms with decimals (type in)
– o -2.1z – 8z = 2
• Combining like variable terms and a constant with decimals (no type
in)
– o 8.1 + 7.4x + 3.7x = 4.6
• Combining like variable terms and a constant with decimals (type in)
– o -6.6z – 2.8 – 6.7z = 2.8
Target Material
Linear equations and the distributive property
• Solving Linear Equations (Type In)
– o 7 + 2x = -3
• Using Multiplication and Integers (No Type In)
– o 4(z - 3) = 3
• Using Multiplication and Integers (Type In)
– o -6(z – 6) = 10
• Using Multiplication and Decimals (No Type In)
– o -4.5 = -2.6(z + 2.9)
• Using Multiplication and Decimals (Type In)
– o 8.3(z – 5.9) = 6.8
• Using Multiplication and Large Decimals (No Type In)
– o -43.54 = -16.63(z + 24.90)
• Using Multiplication and Large Decimals (Type In)
– o -51.78(w + 97.21) = -11.63
• Using Division (No Type In)
Experimental Conditions
Kanawha County School District
3 Conditions divided by class (4 classes for
each condition):
Discovery, Instruction, Standard
Presented in line with curriculum.
Experimental Conditions
Discovery
Direct Instruction
Current tutor
Hints
“How” hints only
(on general
interface)
“How” and “why”
hints available
“How” and “why”,
no changes to
2011 std release
Just in Time (JITs)
Messages
None
Normal
Normal
Step by Step
Guided Problem
“How” hints only
Force step-bystep in the
beginning of new
section
“How” and “why”
Force step-bystep in the
beginning of new
section
Optional
Side by Side
Worked Examples
Side-by-side
worked example
with history tags
Side-by-side
worked example
with history tags
None
Example Types
• Step-by-Step Guided Problem
One per section
– selected based on sections “new” material
Students simply follow directions in a step-bystep manner to solve the problem
• Side-by-Side Worked Examples
Already solved problems appear next to a
target problem
Step-by-Step Guided Problem
Instruction
w+6
=
-1
To isolate w, subtract 6 from both
sides. Select “subtract both
sides” and enter 6.
Instruction condition
•Provide “why” and “how” hints
•Directly tell students what to do
Step-by-Step Guided Problem
Discovery
w+6
=
-1
Select an item from Transform
menu and enter a number.
Discovery condition
•Provide “how” hints
•Provide instruction on general
interface
Color-Coding
w+6
-6
w+6–6
=
=
-1
-6
subtract 6 from both sides
-1 – 6
Both conditions
• Provide guided problem
• Align equation on the equal sign
• Coloring variable side of the equation
• Highlighting changes between steps
Wrong Attempts
w+6
+6
w+6+6
=
=
-1
+6
add 6 from both sides
-1 + 6
This is not the best way to solve the
problem. Click “undo” and ask for
hint if you need help.
Both conditions
•Reduce off-path steps
using “undo” button.
Side by Side Worked Example
Instruction & Discovery
Example
w+6
+6
=
-1
+6
.
.
add 6 from both sides
w+6+6
=
-1 + 6
Problem
8 = x-2
combine like terms
w
=
5
Both conditions
• Provide already solved
problems (screen-shot)
• Selected from the pool of
problems in the same section
Post-Test
2 types of test questions:
(paper-based, within subject)
-Plain equations
-Equations with solver interface
scaffolding