Self-diagnosis, Scaffolding and
Transfer: A tale of two Problems
A. Mason1, E. Cohen2, C. Singh1 and E.
Yerushalmi2
1University
of Pittsburgh
2Weizmann Institute of Science, Rehovot, Israel
PERC 2009
Research Design
Goal: recitation sections deliberately prompted to selfdiagnose, given varying levels of support
– Who will perform the best?
1st stage: Students attempt a quiz problem,
2nd stage: Intervention-Self-diagnosis with alternative supports
Control-no SD TA outline
TA discuss
+diagnosis
solution, (A)
Rubric (B)
3rd stage Post: Solving paired quiz
Sample
Solution
(C)
problem
Text+
note
Book (D)
100 students
28 students
24 students
31 students
First-semester algebra-based introductory physics
1 instructor, 2 TAs
Research Rationale
1) What are students able to diagnose if deliberately
prompted to self-diagnose given alternate supports?
2) What is the effect of SD on consecutive problem
solving?
Assumption: high-performers are the ones which learn
from their mistakes. If we can get low-performers
to learn from their mistakes,
3) Do we reduce the gap between low- and highperformers?
1st study
Non-Conventional problem,
Paired midterm problem given
a few days after selfdiagnosis
Quiz 6 and its Post (midterm II)
A friend told a girl that he had heard that if you sit on a scale while riding
on a roller coaster, the dial on the scale changes all the time. The girl
decides to check the story and takes a bathroom scale to the amusement
park. There she receives an illustration (provided), depicting the riding
track of a roller coaster car along with information on the track (the
illustration scale is not accurate). The operator of the ride informs her
that the rail track is smooth, the mass of the car is 120 kg, and that the car
sets in motion from a rest position at the height of 15m. He adds that point
B is at 5m height and that close to point B the track is part of a circle with a
radius of 30m. Before leaving the house, the girl stepped on the scale which
indicated 55kg. In the roller coaster car the girl sits on the scale. Do you
think that the story she had heard about the reading of the scale changing
on the roller coaster is true? According to your calculation, what will the
scale show at point B?
Girl on rollercoaster going over a
circular bump – at peak of bump,
how does weight change on a
scale?
A family decides to create a tire swing in their back yard for
their son Ryan. They tie a nylon rope to a branch that is
located 16 m above the earth, and adjust it so that the tire
swings 1 meter above the ground. To make the ride more
exciting, they construct a launch point that is 13 m above the
ground, so that they don't have to push Ryan all the time.
You are their neighbor, and you are concerned that the ride
might not be safe, so you calculate the maximum tension in
the rope to see if it will hold. (a) Where is the tension
greatest? (b) Calculate the maximum tension in the rope,
assuming that Ryan (mass 30 kg) starts from rest from his
launch pad. Is it greater than the rated value of 750 N? (c)
Name two factors that may have been ignored in the above
analysis, and describe whether they make the ride less safe
or more safe.
2nd study
Conventional problem,
Paired midterm problem given a
month after self-diagnosis
Quiz 7 and its Post (midterm III)
You are helping a friend prepare for the next skate board exhibition.
Your friend who weighs 60 kg will take a running start and then jump
with a speed of 1.5 m/s onto a heavy duty 5 kg stationary skateboard.
Your friend and the skateboard will then glide together in a straight
line along a short, level section of track, then up a sloped concrete
incline plane. Your friend wants to reach a minimum height of 3 m above
the starting level before he comes to rest and starts to come back
down the slope. Knowing that you have taken physics, your friend wants
you to determine if the plan can be carried out or whether he will stop
before reaching a 3 m height. Do not ignore the mass of the
skateboard.
Boy jumping on skateboard going
over a inclined plane –what height
will he reach? (i.e. solve for max
height)
Fred Flintstone just got off work, and exits in his usual way, sliding
down the tail of his dinosaur and landing in his car (see Figure). Given
the height of the dinosaur (h=10 m), it's not hard to calculate his
speed v as he enters his vehicle.
Conservation of energy yields the following equation: mgh=1/2 mv2,
where m=100 kg is Fred's mass and v is his speed. Algebraic
manipulation yields v=sqrt(2gh)=14 m/s. Judging from the picture
taken in Figure 1E, the angle at which Fred enters the car is
approximately 450. (a) If the mass of the car is M=200 kg, find the
speed with which Fred is driving in the last frame (Figure 1F), assuming
he hasn't used his feet to pedal. (Remember also that there are no
fossil fuels since there are no fossils yet.)
(b) Assuming that there is no friction or air resistance, determine the
maximum height H that Fred and his car can travel without extra
pushing.
Both pairs exhibit similar general procedure
(principles/intermediate variables), different surface
features
Analysis Tools-Analysis Rubric
I-quiz S-quiz
(grade
Generic
Physics
Invoking
Appropriate
principles
EC, MC, Newton second law…
Justification
Justify EC and MC…
Compactness
Applying
Presenting reasoning
Specific
EC, MC, Newton second law…
Description
Drawing
Knowns
plan
Target +
intermediate
variables
checking
FBD, masses & velocities
before and after collision,
velocities and heights or Ek
and Ep before and after going
up an incline…
v
a
h…
Units,
limit case
Total Scores (Phy. Score, Pres. Score)
I-SD
of (grade of quiz (researcher’s
quiz
solution as
grade of the
solution
diagnosed by diagnosis done
given by
student)
by the
researcher
student)
)
Valid:
focus on students’
diagnostic ability
Versatile:
generic/specific
Reliable: Inter-rater
reliability > 80%
Research questions and expectations:
Research
Questions
Inter group comparison:
Given alternative supports,
 how will external support affect
self-diagnosis?
 how will it affect consecutive
problem-solving
Intra group comparison:
How does the self-diagnosis
affect the gap between low
and high achieving students?
 Do low achievers succeed in
diagnosing themselves?
 Do they perform better in
consecutive problem-solving?
Expectations
More external support
 better self-diagnosis
 better performance on
consecutive problem-solving
The self-diagnosis task will be
successful in reducing the gap
between low and high
achieving students
Reservations:
If self-diagnosis
is NOT
MEANINGFUL
Then between-group differences
in self-diagnosis performance
won’t be reflected in consecutive
problem-solving
The gap between low and high
achievers will remain the same
in consecutive problem-solving
What is a MEANINGFUL self-diagnosis?
THEORY (Chi, 2000): Students learn from solved examples by providing
self-explanations. The student is expected:
Instructor’s
mind:
“Scientific
Mental Model”
COMPARISON
difference
b) to acknowledge
conflicts between
their mental model
and the sample
solution, leading to
self-repair of a
flawed mental
model.
Sample
Solution
Significant
a) to compare and
realize significant
differences
between the two
solutions;
CONFLICT
Self-repairing
flawed mental model
Student’s
Solution
Student’s mind:
“Flawed Mental
Model”
How does a NON-MEANINGFUL self diagnosis
differ from a MEANINGFUL one?
1. “Meaningful intervention”: Students’ scoring on the diagnosis of
their mistakes is independent of their prior knowledge, and involves
self-repairing their mental model
Only in this case do we expect better self-diagnosis to lead to better
performance on consecutive problem-solving, thus reducing the gap
between low and high achievers.
VS.
2. “Superficial intervention“: Students’ scoring on the diagnosis of
their mistakes is independent of their prior knowledge, yet students
do not involve self-repairing their mental model
3. “Weak intervention“: Students’ scoring on the diagnosis of their
mistakes depends on their prior knowledge.
Expectations - intra group comparison
Intervention
Meaningful Superficial
Correlation
Pre vs. SD
SD vs. Post
Pre vs. Post
Weak
Control
(Supports weak
students in
improving their SD
performance.
Involves
meaningful learning
leading to
transfer.)
(Supports weak
students in
improving their SD
performance,
However, does not
involve meaningful
learning leading to
transfer.)
(Does not
support weak
students to
improve their SD
performance)
N/A
N/A
Positive
N/S
Positive
N/S
N/S
N/S
Positive
Positive
Positive
Positive
Inter group findings
Intervention
1st study
SD mean (Std. Err)
(non
P value
conventional
Post mean (Std. Err)
problem)
2nd study
Outline +
Rubric (B)
Sample
solution (C)
Minimal
guidance (D)
0.73 (0.05)
0.57 (0.05)
0.24 (0.06)
B>C>D (ANCOVA based on quiz physics)
0.53 (0.05)
0.33 (0.05)
0.47 (0.06)
P value
C differs from B and D (ANCOVA based on quiz
physics)
SD mean (Std. Err)
0.56 (0.06)
(conventional
P value
problem)
0.62 (0.06)
0.61 (0.05
No difference between groups (ANCOVA based
on quiz physics)
post mean (Std. Err)
0.66 (0.04)
P value
No difference between groups (ANCOVA based
on quiz physics)
0.72 (0.06)
0.76 (0.04)
The inter group findings suggest:
How do alternative external supports affect selfdiagnosis?
 External support makes a difference in a non
conventional problem, but not in a conventional one.
e.g. Text and notes are enough for group (D) to perform SD in quiz 7, but not
in quiz 6, where to self-diagnose students needed sample solution and rubric.
How do alternative external supports affect
consecutive problem-solving?
 It seems it doesn’t:
e.g A) SD average grade for group B in quiz 6 was much better than that for
the other groups, but their post average was comparable to group D.
B) SD average grade for group D in quiz 7 was the same as group C, and
slightly better that that for group B (not significantly). D got the highest
average in their post, although this average was comparable to groups B and C.
Intra group findings,
2nd study group D
Control
A, A’
intervention
Weak
Superficial
Meaningful
Correlation PreSD
N/A
N/S
Correlation SDPost
N/A
Positive
corr=0.53, p
value<0.05)
Correlation PrePost
Positive
0.44/0.35
p<0.05
N/S
Intra group findings,
1st study all groups, 2nd study all groups but D
Control
A, A’
intervention
Weak
Superficial
Meaningful
N/S
Correlation PreSD
N/A
N/S
Correlation SDPost
N/A
N/S
Correlation PrePost
Positive
0.44/0.35
p<0.05
?
N/S
The intra group findings suggest:
Group D, quiz 7
All other attempts, all
groups
D performs a
meaningful diagnosis
PUZZLE – no match to
expectations!
Pre/Post correlation is not
significant (midterm
performance is independent of
prior knowledge), while it is
positive for control
- On one hand, the intervention
seems to be meaningful (Pre-post
positive for control & none for
intervention groups)
-
- On the other hand, it seems
- SD/Post correlation is positive superficial, as for most groups
(midterm performance depends there is no correlation SD-Post.
on SD performance)
Possible resolution for puzzle
group D
Self diagnosing with minimal
help (D, 2nd study)
=
Meaningful SD
Meaningful SD
D had to struggle to find a related sample solution
in the text, which they could in the 2nd study
(conventional)  meaningful diagnosis
Possible resolution for puzzle
groups B and C
Self diagnosing with
maximal help (B,C)

Meaningful SD
Superficial SD ???
It is hard to differentiate between meaningful and
superficial diagnosis as the sample solution allows to
diagnose “without struggling”
Possible resolution for puzzle
What is unique for group D?
Meaningful self diagnosis involves [Chi]:
Stage a) comparing and realizing significant differences
between the two solutions;
Stage b) acknowledging conflicts between a flawed mental
model and the sample solution;
Student rarely make explicit remarks reflecting stage b,
thus, the question is whether we can conclude from remarks
reflecting stage a - difference between the two solutions that a meaningful self diagnosis took place,
We claim that we can do so only for students who received
minimal support for SD.
Possible resolution for puzzle
What is unique for group D?
Students in groups B and C could compare their solutions
to the sample solution provided and with minimal cognitive
engagement state: "I did not do this equation".
Yet, in group D, students who were able to diagnose must have
been cognitively engaged as they had to search on their own
for a solution related to the problem they were trying to
Diagnose.
Therefore, for group D the grade indeed indicates how
meaningful the diagnosis is.