SREE Fall 2011
The Effects of Feedback During
Exploratory Math Practice
Emily R. Fyfe & Bethany Rittle-Johnson
Vanderbilt University
Marci S. DeCaro
University of Louisville
1
How do children learn best?
Two schools of thought have emerged
• Advocates of Direct Instruction
(Kirschner, Sweller, & Clark, 2006)
• Advocates of Discovery Learning
(Bruner, 1961; Kuhn, 1989; Piaget, 1973)
2
An Integrated Perspective
No need for this strict dichotomy
“A mixture of guidance
and exploration is
needed” (Mayer, 2004)
“Proponents of constructivism
and direct instruction…each
have something to learn from
the other” (Rieber, 1992)
“Characterizing discovery and
direct instruction as diametrically
opposed…has done a disservice
to both approaches” (Wilson,
et al, 2010)
“There’s a place for both
direct instruction and
student-directed
inquiry” (Kuhn, 2007)
3
Combining Instruction and Discovery
Exploration prior to instruction facilitates learning
(DeCaro & Rittle-Johnson, 2011; Schwartz & Bransford, 1998)
Hints or coaching during problem solving is better than
pure problem solving alone
(Mayer, 2004)
Recent meta-analysis indicates that guided discovery is
better than unassisted discovery or direct instruction
(Alfieri et al, 2010)
4
Questions remain…
Which elements of instruction are most suitable to
incorporate within exploration?
For whom is guided exploration most advantageous
and why?
5
Feedback During Exploration
Feedback represents one element of instruction
that may be particularly effective in combination
with exploration
What is Feedback?
• Any information that the learner can use to
confirm, reject, or modify prior knowledge
• Accuracy information, hints, etc.
6
Types of Feedback
Outcome Feedback
•Provides information about
learner’s answer
Strategy Feedback
•Provides information about
how answer was obtained
•Used extensively
•Only examined in a few
previous studies
•Related to strong, positive
effects in past work compared
to no feedback
•Better than outcome
feedback in terms of strategy
selection
(Ahmad, 1988; Kluger & DeNisi, 1996; Luwel et al., 2011)
7
Why feedback?
May reduce disadvantages of discovery by guiding
the learner’s search for information
Helps identify errors and encourages search for
plausible alternatives (e.g., new strategies)
(Hattie & Timperley, 2007; Mory, 2004)
But…
• Past research indicates variable efficacy
(Kluger & DeNisi, 1996)
• May only benefit a subset of learners
8
What about prior knowledge?
Expertise reversal effect
• Instructional technique is effective for novices, but
loses its benefits for high-knowledge learners
(Kalyuga, Ayres, Chandler, & Sweller, 2003)
• Example: worked examples vs. problem solving
Low knowledge learners benefit from more external
guidance; high knowledge learners benefit from less
Perhaps feedback during exploration only helps
children who have low prior knowledge
9
Why the reversal?
Due to existing cognitive resources
(Paas, Renkl, & Sweller, 2003)
• Novices lack schemas; need external guidance to
reduce cognitive load
• High-knowledge learners have schemas;
additional guidance creates more cognitive load
10
Goals of this study
Examine the effects of feedback during exploratory
math practice for children with varying levels of
prior knowledge
Specifically
• Compare feedback vs. no feedback
• Compare outcome vs. strategy feedback
• Look at effects of prior knowledge
11
Hypotheses
Ho 1:
Feedback > No Feedback
Ho 2:
Strategy Feedback > Outcome Feedback
Ho 3:
Feedback better for children with low prior knowledge
12
Domain: Mathematical Equivalence
Concept that two sides of an equation represent
the same amount and are interchangeable
• Commonly represented by equal sign (=)
3+7+8=3+_
6+4=_+8
13
Why Study Math Equivalence?
Fundamental concept in arithmetic and algebra
Very difficult for children in U.S.
• Interpret equal sign as an operator symbol that
means “the total” as opposed to relational symbol
(McNeil, 2008; Rittle-Johnson & Alibali, 1999)
• In one study, only 24% of U.S. children in 3rd and 4th
grade solved math equivalence problems correctly
(McNeil & Alibali, 2000)
14
Participants
Worked with 91 children (2nd & 3rd grade)
-
M age = 8 yrs, 7 mo
53 females, 38 males
45% white, 40% black, 15% other
47% receive free or reduced lunch
15
Design and Procedure
Session 1: Pretest (~25 minutes)
• Excluded if score >80% on pretest measures
Session 2: Intervention & Posttest (~50 minutes)
Session 3: Two-week Retention Test (~25 minutes)
16
Tutoring Intervention
Exploratory Practice
• Attempt to solve 12 math equivalence problems
• Randomly assigned to 1 of 3 conditions
• No Feedback (n = 31)
• Outcome Feedback (n = 32)
• Strategy Feedback (n = 28)
Midtest
Brief conceptual instruction
(DeCaro & Rittle-Johnson, 2011)
17
Exploratory Practice
Find the number that goes in the blank.
3+4+8=3+☐
How did you solve that problem?
No Feedback: “OK, let’s move on to the next problem.”
Outcome Feedback: “Good try, but that’s not the correct answer. The
correct answer is 12.”
Strategy Feedback: “Good try, but that’s not a correct way to solve
that problem.”
18
Assessment of Math Equivalence
Procedural Knowledge
• Use correct strategy to solve problems
7+6+4=7+_
Used at Pretest,
Midtest, Posttest,
& Retention Test
6-4+3=_+3
Conceptual Knowledge
• Understand concept of equivalence
What does the equal
sign mean?
4+8=8+4
True or False?
(Rittle-Johnson, Matthews, Taylor, & McEldoon, 2011)
19
Analysis & Results
Contrast-based ANCOVA (West, Aiken, & Krull, 1996)
• Two contrast-coded condition variables
-
Feedback (no feedback vs. two feedback conditions combined)
Feedback Type (outcome feedback vs. strategy feedback)
• Two condition x prior knowledge interactions
• Three covariates
Interaction follow-up
• Categorize as low vs. high knowledge (median split)
• Simple main effects of condition
20
Procedural Knowledge
Repeated Measures ANCOVA: Midtest, Posttest, and Retention Test.
21
Procedural Knowledge
Overall feedback x prior knowledge interaction, F(1, 83) = 7.05, p = .01
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 3.84, p = .05
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.61, p = .04
*
*
22
Procedural Knowledge Results
Expertise reversal effect
Feedback during exploratory math practice is more
beneficial than no feedback, but only for children
with low knowledge
For children with high prior knowledge, the reverse
is true; they benefit more from no feedback
23
Intervention Activities
Subjective Cognitive Load (NASA-TLX, Hart & Staveland, 1988)
• I had to work hard to solve those problems.
• I was stressed and irritated when I had to solve
those problems.
• Mean rating on agreement scale from 1 to 5.
Problem-Solving Strategy Use
• Variability of correct & incorrect strategies
24
Cognitive Load
*
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 1.15, p = .29
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 6.05, p = .02
25
Cognitive Load
Supports expertise reversal effect explanation
Feedback may have hurt high-knowledge learners’
performance because of increased cognitive load
An effect not found for low-knowledge learners
26
Strategy Coding Scheme
Strategy
Sample explanation (4 + 5 + 8 = __ + 8)
Correct Strategies
Equalize
I added 4, 5, and 8 and got 17. And 9 plus 8 is 17.
Add-Subtract
I added 4, 5, and 8 and got 17. And 17 minus 8 is 9.
Grouping
I took out the 8’s and I added 4 plus 5.
Incorrect Strategies
Add-All
I added the 4, 5, 8 and 8.
Add-to-Equal
I just added the first three, the 4, 5, and 8.
Carry
I saw a 4 here, so I wrote a 4 in the blank.
Ambiguous
I used 8 plus 8, and then 5.
27
Perseveration
Perseveration = Using the same incorrect strategy on all the problems.
*
* P = .01
28
Incorrect Strategy Variability
*
*
Ranged from 0 to 5.
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 5.66, p = .02
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.56, p = .04
29
Correct Strategy Variability
*
Ranged from 0 to 3.
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.76, p = .03
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 0.16, p = .90
30
Strategy Variability
For children with low prior knowledge, feedback
prevented perseveration and led to the
generation of diverse strategies
-
May explain why these children learned more when
they received feedback than when they did not
For children with higher prior knowledge, feedback
led to the generation of more incorrect, but not
correct strategies
-
May help explain why feedback had negative impact
31
Summary
Feedback led to higher procedural knowledge of
math equivalence than no feedback, but only for
children with low prior knowledge
For children with high prior knowledge, no
feedback was better
No differences between outcome feedback and
strategy feedback
32
Implications
Theoretical
• Extends expertise reversal effect to feedback
• Clarifies research on discovery learning
Practical
• Pay more attention to when you give feedback
during tutoring and teaching
33
Thank You
Children’s Learning Lab
Bethany Rittle-Johnson
Marci DeCaro
Laura McLean
Maryphyllis Crean
Lucy Rice
Funding Sources
ExpERT Training Grant,
through IES to Fyfe
NSF CAREER grant to
Rittle-Johnson
34
Instruct vs. Discover
Direct Instruction
Discovery Learning
•Told or shown how to solve the
problems
•Explore the problems on your
own with no guidance
•Provides structure and reduces
task ambiguity
•Find target information and new
strategies independently
•But…
•But…
• Limits self-discovery
• Might limit learner
engagement
• Overwhelms working memory
• Might never locate target info
or invent correct strategy
(Kirschner, Sweller, & Clark, 2006; Mayer, 2004; Sweller, 1988)
35
Exploratory Practice
Children’s performance on 12 problems during
intervention (Percent Correct)
36
Instruction
Let’s take a look at this problem.
3+4=3+4
There are two sides to this problem, one on the left
side of the equal sign and one on the right side of
the equal sign…
The equal sign means that the left side of the equal
side is the SAME AMOUNT AS the right side of
the equal sign. That is, things on both sides of the
equal sign are equal or the same.
37
Conceptual Knowledge
Feedback type x prior knowledge interaction, F(1, 83) = 4.82, p = .03.
Low knowledge: No effect of feedback type, F(1, 83) = 0.51, p = .48
High knowledge: Main effect of feedback type, F(1, 83) = 8.21, p = .005
38
Intervention – Strategy Use
Incorrect
Correct
*
*
*
Note. Differences are between no feedback condition and two feedback conditions
combined: * p < .05
39
Strategy Variability
*
*
Note. Difference is between no feedback condition and two feedback conditions
combined: * p < .001
40