Amsel
1
I. Significance
It is widely accepted that a central goal of education is to promote good reasoning skills
in students (Bruer, 1993; Carmichael, 1981; Chance, 1986; Baron, 1985, 1988; Brown, 1990; De
Bono, 1983, 1985; Greeno & Goldman, 1998; Hogarth, 2001; Kuhn, 2005; Nickerson, Perkins,
& Smith, 1985; Perkins, 1992; Resnick, 1987; Resnick & Klopfer, 1989; Schoenfeld, 1992).
However, there is little theoretical consensus about the nature of good reasoning, leaving unclear
the pedagogical prescriptions for promoting it (Perkins et al., 1993). One source of theoretical
contention is whether or not people acquire the skills to reason in a fundamentally rational
manner spontaneously or through training (see reviews by Baron, 1998; Cohen, 1981; Evans &
Over, 1996; Moshman, 2004; Stanovich & West, 2000).
Recent theoretical developments in Social and Cognitive Psychology have offered new
perspectives on the rationality issue. "Dual Process" theories (of which there are many versions,
c.f., Chaiklen & Trope, 1999) deny that the mind is either rational or irrational and instead hold
that it is both. Dual process theories propose that two cognitive systems are simultaneously
involved in the processing of any information (c.f., Bruner, 1986; Evans & Over, 1996; Hogarth,
2005; Klaczynski, 2004; Kirkpatrick & Epstein, 1992; Reyna, & Brainerd, 1995; Stanovich,
1999; Stanovich & West, 2000). In Epstein’s version (Kirkpatrick & Epstein, 1992; Epstein &
Panici, 1999), which is adopted in this proposal, the dual processes include an Experiential
System, which is automatic, spontaneous, largely unconscious, and heuristic, and an Analytic
System, which is effortful, systematic, largely conscious, and logical. These processes are
believed not only to act in parallel but also to interact, with Experiential processing usually
serving as the default system–the system on which people typically rely in everyday judgment
and decision situations (e.g., Evans & Over, 1996; Klaczynski, 2004; Sloman, 1996; Stanovich
& West, 2000). However, situational cues and motivational dispositions may determine which
form of processing is dominant in particular circumstances.
A. Dual Process Theory and Developmental and Educational Psychology
The implications and applications of Dual Process theory for Cognitive Developmental
and Educational psychology have not been well explored because of conceptual incompatibility
between the theory and assumptions about the nature of development and learning (Amsel, 2004;
Brainerd & Reyna, 2004; Klaczynski, 2004). Development and learning has long been
theoretically described as a progression leading from relatively unreflective, automatic, and
intuitive (experiential) processing of information to a more reflective, effortful, and systematic
(analytic) processing. For example, constructivist approaches to learning mathematics (e.g.,
Piaget and Vygotsky) assume that younger and less mathematically sophisticated students may
respond experientially on mathematical problems and only later respond analytically, as they
grasp formal and normative basis of mathematics (Greenes, 1995).
Dual Processing theory denies that development or learning involves the simple
replacement of the experiential by the analytic processing system and rather presumes the two
forms of processing interact as both develop in tandem. It is argued that the two systems develop
concurrently and that as preadolescents, adolescents, and adults come to more strongly rely on
default experiential processing, they simultaneously acquire the ability to potentially override the
Amsel
2
experiential system and rely on the analytic system (Brainerd & Reyna, 2004, Klaczynski, 2004;
Stanovich & West, 2000). The result of these two developmental trends – greater reliance on
contextualized and automatic experiential processing and greater potential for decontextualized
and effortful analytic processing – makes for predictions of counterintuitive developmental
trajectories for many developing phenomena under certain conditions. For example, dual process
accounts have confirmed counterintuitive theoretical predictions regarding developmental
sequences in such domains as memory development (Brainerd & Reyna, 2001), risk taking
(Reyna & Ellis, 1994; Reyna, 2004); reasoning (Amsel, Close, Sadler, & Klaczynski, 2005a;
Klaczynski, 2001a, 2001b, 2004; Markovits & Barrouillet, 2002), decision-making (Amsel,
Cottrell, Sullivan, & Bowden, 2005; Klaczynski, 2004; Jacobs & Klaczynski, 2001), hypothesis
testing (Klaczynski, 2000; Stanovich & West, 1997) and learning (Dixon & Moore, 1996;
Moore, Dixon & Haynes, 1991). For example, Dual Processing theory better captures many
phenomena associated with the acquisition of expertise. Novices may rely exclusively on
experiential processing in problem-solving due to a lack of domain-specific knowledge and
analytic strategies. However, contrary to standard developmental theory, experts may also rely
on experiential processes. The experts learned analytic strategies which were deployed
effortfully at first but were later automatized as experiential processes (Bereiter & Scardamalia,
1993; Brainerd & Reyna, 2004; Chi, Glaser & Farr, 1988; Sternberg & Frensch, 1992).
B. Dual Process Regulation (DPR)
Another reason for the limited impact of dual process theories in Developmental and
Educational psychology is that little is known theoretically and empirically about the
spontaneous development and trainability of the regulatory mechanisms governing the
experiential-analytic interactions. Such a regulatory mechanism is the basis for individuals
determining which system to rely on for a given problem. Although the experiential processing
system may be the default, responses on a given task may at times be a product of deliberately
selecting the outcome associated with one processing system, particularly when each system is
activated and the outcomes of each are conflicting. Hogarth (2005) specifically argues that there
are trade-offs between potential sources of errors between the systems resulting in certain
systems being more valid in response to certain tasks. In his account, valid decisions would
require relying on the processing system in a particular context which minimizes the errors
relative to the task demands. For example, analytic-based deliberation has been shown to
undermine experiential-based intuitive judgments about values or preferences (Wilson &
Schooler, 1991). On a similar point, Amsel, Cottrell, Sullivan, & Bowden (2005) found that
college students (although interestingly not preadolescents) would bypass the result of the
analytic-based subjective expected utility processing strategy in order to respond on the basis of
experientially-based analysis of anticipated regrets.
Although Hogarth’s (2005) analysis suggests that regulation of dual processes is
necessary, he provides no mechanism by which such regulation is accomplished. Klaczynski
(2004) discusses the nature and development of dual process regulation (DPR), a process he
labels “metacognitive intercession.” He notes that such skills enable the determination and
selection of appropriate task strategies to solve tasks which have been appropriately
decontextualized. This stands in contrast to the experiential system which involves the automatic
activation and application of memory-based procedures (e.g., heuristics) to highly contextualized
Amsel
3
task representations. At certain critical phases of problem solving, the outcomes of the analytic
processing system may be compared with those of the experiential system, while both exist in
memory. This critical phase is defined by the time period in which the outcomes of the
experiential system can be inhibited but remain in working memory in order to be compared with
the outcomes of the analytic system.
C. DPR Skills and Metacognition in Mathematics Cognition and Achievement
From this account, Dual Process Regulatory (DPR) skills would seem to share many
features in common with general metacognition. Metacognition is the general ability to be aware
of and have knowledge about one’s mental processes such that one can control them to a desired
end, evaluate the outcome, and reallocate resources should the situation require it (Flavell, 1979;
Brown, 1978; Tobias & Everson, 2000). There is evidence of the impact of metacognitive
training on mathematical thinking and achievement in middle-and high school students’ algebra
and geometry classes (Kramarski, 2004; Mevarech & Kramarski 2003; Kramarski, Mevarech, &
Lieberman, 2001; Perrenet, & Wolters, 1994; Schoenfeld, 1992; Swanson, 1990; Teong, 2003).
Mathematics is a particularly interesting domain in which to explore the regulation of dual
processing system, being a formal and normative domain. In the language of Hogarth, the task
domain is one in which valid judgments are likely the result of reliance on analytic over
experiential processes, particularly for student novices because their experiential processes are
profoundly biased and error-prone (Bell, 1976; Chazen, 1993; Segal, 1999). Experts in the
domain can rely on the validity of the outcomes of experiential processes (Sriraman, 2004),
having refined those intuitions in environments providing extensive and valid feedback (Hogarth,
2005).
Although overlapping, there are two ways in which DPR skills may implicate a different set of
processes and abilities than do general metacognitive skills. In one sense, DPR is a broader
construct than metacognition because DPR is not only a cognitive but also a motivational
construct implicating cognitive and personality dispositions valuing deep, as opposed to
superficial, processing of information (Biggs, Kember, & Leung, 2001; Cacioppo, Petty,
Feinstein, & Javis, 1996; Epstein, Denes-Raj, & Heier, 1996; Klaczynski, 2004; Stanovich &
West, 2000). Such dispositional learning styles may contribute to students’ success in their
tendency not only to acquire but also to use DPR skills. In another sense, DPR may be a
narrower phenomenon that is subserved by more specific cognitive mechanisms than
metacognition. Metacognition as a construct includes understanding the knowing process itself
(Hofer, 2004; Kuhn, 2001, Pintauch, 2002), whereas no such knowledge is implicated in DPR
skills. The latter are used exclusively to intervene in the response generation process,
specifically when conflicting response options generated by analytic and experiential processes
are available for inspection and comparison. Such skills have been associated with such
processes as working memory capacity (Barrett, Tugade, & Engle, 2004; Smith & DeCoster,
2000) and inhibitory control (Klaczynski, 2004). These processes may be best thought of as
specific aspects of executive function (EF) which specify neurological systems subserving
problem-solving and decision-making processes (Baddeley, 1996; Bull & Scerif, 2001; Carlson,
Moses, & Hix, 1998; Dempster, 1992; Denckla & Reiss, 1997; Norman & Shallice, 1986;
Tranel, Anderson, & Benton, 1994; Zelazo & Meuller, 2002).
Amsel
4
The above review suggests that DPR skills may play a small but critical role in the effectiveness
of metacognition training in promoting mathematical cognition and achievement. General
dispositional and specific neurocognitive factors may contribute to whether an individual
engages in DPR during the time period defining the critical phase. In support of this view,
Peterson, Pihl, Higgins, Seguin, & Tremblay (2003) report that assessments of executive
function and personality each predicted middle school and high school math grades over a 6-year
period above and beyond the variance accounted for by IQ. Similarly, neuropsychological
measures of executive function predicted math performance independently of memory capacity
(Bull & Scerif, 2001) and calculation ability (Kwon, Lawson, Chung, & Kim, 2000). These
findings serve as indirect evidence that mathematical cognition and achievement are predicted by
the kinds of specific neuropsychological abilities implicated by DPR skills.
D. Measuring DPR Skills: The Ratio Bias Task
There is evidence that DPR both develops spontaneously and has the potential to be
trained. One line of research on which this conclusion is based is a simple mathematical task
involving the comparison of ratios. The Ratio Bias task is a computationally easy but intuitively
error-prone mathematical problem in which participants are offered a choice between two equal
gambles, one with a 1/10 and the other with a 10/100 chance of winning (Kirkpatrick & Epstein,
1992). Most college students prefer the 10/100 over the 1/10 gamble, an example of the primacy
of experiential processing of the absolute number of winners over the analytic processing of ratio
information. In one of the first developmental studies on the topic, Klaczynski (2001b) found
increases in correct responses (i.e., judgments of no preference) on the ratio bias task from
middle school to college, but only a minority of the college students responded correctly (see
also Alonso-Berrocol & Fernandez, 2003). Nonetheless, the finding of age-related changes was
attributed to students’ growth of dual process regulatory (DPR) skills to respond on the basis of
the analytic instead of the experiential system. Klaczynski (2001b) further found that the
tendency to respond experientially rather than analytically was reduced by merely altering the
perspective participants adopted when processing the information. When told to respond from
the perspective of a perfectly logical person, participants of all ages responded correctly more
often than when responding from their own (Self) perspective. The perspective effect on the ratio
bias task has been replicated with American college and high school students (Amsel, Close,
Sadler, & Klaczynski, 2005a, 2005c; Epstein & Pacini, 2000) and Spanish high-school students
(Alonso & Fernadez-Berrocoll, 2003).The findings suggest that participants’ responses on the
ratio bias are not stable, but task features can elicit more experiential or rational processing.
In recent work, Amsel and his colleagues have explored the nature of students'
underlying competence for regulating dual experiential and rational processes and its implication
for their mathematical cognition and learning (Amsel, Close, Sadler, & Klaczynski, 2005a,
2005b, 2005c, Bates & Amsel, 2005; Leavitt, Bates, & Amsel, 2005). Amsel et al. (2005a)
examined not only college students’ judgments on the ratio bias task, but also their evaluation of
each response option as to whether it was based on an analytic process. The Ratio-Basis
Judgment (RB-J) task is the same task used by Klaczynski (2001b), in which participants were
presented with two gambles (1/10 vs. 10/100), told that they were equal, and asked to choose
one, the other or to express no preference. In the Ratio-Bias Evaluation (RB-E) task, participants
were asked to evaluate how certain they were that choosing each response option (1/10, 10/100,
Amsel
5
and no preference) reflected a product of analytic processing of the information (defined as a
“logical, thoughtful, and mathematically sound analysis of the situation”) as opposed the product
of an experiential processing of it (defined as “automatic reaction to or gut feeling about the
situation”). The RB-E task assesses whether participants correctly recognize the products of
Analytic processing, even though they may prefer the product of Experiential processing on the
RB-J task.
Amsel et al. (2005a) found that most participants had a preferred gamble on the RB-J task
and few were able to appropriately evaluate the rational response option. As evidence of the
validity of the RB-E task as an assessment of regulatory competence, Amsel et al. (2005a) found
that college students’ responses on RB-E task (whether that could identify a rational response
option) better predicted their actual gambling behavior and decision-making than did their
responses on the RB-J task (whether they had a preferred gamble). In a series of follow-up
studies, college students’ RB-E performance was shown to be relatively stable over variations in
the Self vs. Logical Person perspectives (Bates & Amsel, 2005; Leavitt, et al., 2005), the ways of
framing the task (i.e., winning vs. avoiding losing) (Leavitt et al., 2005), and the gambling ratios
(1/10 vs. 10/100; 9/10 vs. 90/100) (Leavitt & Amsel, 2005), although, in each case, participants’
judgments on the RB-J task were affected by the manipulation. That is, while task features may
cause students to prefer one response option over another, DPR competence remained the same.
Together, these data provide validity and reliability for the measurement of students’ DPR
competence.
While stable, students’ DPR competence varied with age and mathematics expertise.
Amsel et al. (2005c) and Leavitt et al. (2005) compared and contrasted the ratio-bias (RB-J and
RB-E) task performances of college pre-algebra (Remedial Math) students (M=23.3 years-old) to
middle school students (M=13 years-old) being taught the same math content and to college
students in a Statistics course (M=24.6 years) being taught more sophisticated math content.
They found more Competent Regulators (whose performance on the RB-E task was flawless)
among older and more mathematically advanced students (51% of Psychology Statistics, 35% of
college Pre-Algebra students, and 17% of Middle School Pre-Algebra students). Independently
of age, more Competent Regulators (58%) were consistent in their RB-J judgments than other
students (30%), suggesting the importance of DPR skills in mathematical cognition. This
implicates DPR skills as a potential source of difficulties in applying the same mathematical
solutions to logically isomorphic problems (e.g., algebraic expression vs. word problems,
Clement, 1982; Cummins, Kintsch, Reusser & Weimer, 1988; Koedinger & Nathan, 2004). That
is, such difficulties may be seen as reflecting poor regulatory competence to apply the same
analytically-based processing responses on isomorphic task. Also, independently of age, more
Competent Regulators (40%-60% depending on study) expected an A in their math class than did
other students (20%), suggesting the importance of DPR skills in academic performance. These
results suggest that Regulatory Competence in mathematics is related to both age and
mathematical background, and is predictive of mathematical cognition and learning.
E. The Pedagogical Implications of DPR skills.
It is difficult to understand how students without competent DPR skills learn to solve
math problems consistently, much less grasp the normative and formal nature of mathematics.
Amsel
6
Students who are unable to recognize cognitive products of analytic and experiential processes
would seem to have difficulty explicating normative mathematical principles or proofs
(Schonfeld, 1988; Segal, 1999). Pedagogically, it is unclear how much middle school or college
pre-algebra teachers appreciate the limited cognitive regulatory skills of their students. But no
matter what instructors know about their students’ DPR competence, the reality is that math
curricula do not directly train such skills. Indeed, a number of the more recent studies testing the
effectiveness of metacognitive training previous discussed (c.f., Kramski, 2004; Mevarech &
Kramski, 2003; Kramski & Ritkof, 2002) has been run with some form of tutoring control group.
These data suggest that tutoring may be effective in helping students with specific answers to
particular problems, but it may not be sufficient to alter students’ underlying DPR skills.
This pedagogical issues regarding mathematics students’ DPR skills is particularly acute
for college level pre-algebra (remedial math) students. Although the role of DPR skills can not
(yet) be directly implicated as a cause of students’ math cognition and achievement, the
graduation and retention statistics confirm that college remedial math students are vulnerable to
serious academic difficulties. At Weber State University (WSU) 73% of the entering Freshman
class of Fall 2003 either were required to take or took a Remedial Math class (WSU, IR, 2004), a
statistic which is consistent with the freshmen entering the California State system (McClory,
2002). The prerequisite Algebra class for the university Quantitative Literacy requirement has
one of the highest student non-completion rates (including withdrawals, dropouts and failures) in
the university, hovering at 40% (WSU IR, 2004). Students enrolled in a preparatory Pre-Algebra
course for the Math 1010 have similar graduation rates. About 15% of students in remedial math
classes successfully complete the university quantitative literacy requirement in eight semesters
(Jacobson, 2005), and there is no reason to think that WSU is unusual in this regard (National
Center for Educational Statistics, 1996).
These statistics provide an index of the difficulties facing Remedial Mathematics
students. However, they do not provide insight into the cognitive and learning challenges of
these students, which are of central importance for the development of effective curricula and
teaching strategies. Despite their importance, there has been a paucity of cognitive studies of
Remedial Mathematics students and no other study than the ones reported here have examined
these students from a cognitive-developmental perspective. The goal of the present proposal is to
fill the gap in the research and use the results to develop more effective curricular and instruction
methods. The present proposal is to support research exploring the nature of Remedial
Educational students’ dual process regulatory skills from a developmental perspective, assessing
the relation between dual process regulatory skills to more general learning skills, and testing
descriptively and experimentally how such skills affect their mathematical cognition and
achievement. Specifically, four lines of research are proposed. The first explores the relation
between university students’ DPR skills, their relation to IQ, cognitive, neurocognitive,
metacognition, and dispositional variables, and their consequences for academic performance
generally and in mathematics specifically. The second examines the spontaneous development of
DPR skills and the timing of its acquisition on students’ mathematical attitudes and
achievements (Research Line 2). The third research line tests the impact of training in DPR skills
on pre-algebra students’ mathematical cognitions and learning. The results from these lines of
research will inform the development of a stand-alone web-based DPR training for college and
middle-school students (Research Line 4).
Amsel
7
II. Research Narrative
The overall goals of the research proposals are to examine the nature, development, and
trainability of dual process regulatory (DPR) skills and to assess their impact on mathematically
novice students’ mathematical cognition, learning, and achievement. Each of the four lines of
research are complementary, with the first three providing information for the development and
preliminary test of a stand-alone, web-based intervention program to teach middle-school, highschool, and college pre-algebra students DPR skills.
A. Research line 1: The Nature, Development, and Significance of DPR Skills.
A number of assumptions have been made about the development and nature of DPR
skills and their significance for mathematical cognition, learning, and achievement. A central
goal of the first line of research will be to address the development of DPR skills from
preadolescence to adulthood. There is a good deal of evidence of the growth of other regulatory
skills (metacognitive and executive function) during the time periods targeted by this research
(c.f., Kuhn, 2001; Zelazo, Craik, & Booth, 2004). So it would not be surprising that DPR skills
show the same pattern. Evidence of the growth of DPR skills comes primarily from Amsel et al.,
(2005c) who demonstrated that a higher percentage of adolescents are Flawed Regulators and a
lower percentage of them are Competent regulators compared to college students, even those in
the same pre-algebra classes. However, what is less clear is whether the growth of DPR skills is
related to domain-general or domain-specific processes. These processes can be disentangled
through the use of age-by-expertise designs (Krampe, 2002; Krampe, Engbert, & Kliegl, 2002;
Means & Voss, 1985; Schneider, Gruber, Gold, & Opwis, 1993). Although Amsel et al. (2005c)
identified potential age-by-expertise interaction effects in the development of DPR skills, the
study used an incomplete design. Amsel et al. (2005c) used three groups, including younger
novices (Middle School pre-algebra students), older novices (College remedial pre-algebra
students), and older advanced students (College statistics students). Missing in the analysis was
a younger and mathematically advanced group of students. A primary goal of the present
research is to test for age and expertise with a complete design, which will be accomplished by
the inclusion of high-school and college pre-algebra and calculus students. The four cells of
younger and older (2 levels of age) novice and advanced mathematics (2 levels of expertise)
students will complete the factorial design and permit complete analysis of the relation between
age and mathematics background on DPR skills. It is predicted that age effects on the
development of DPR skills will be related to age and mathematics background, independently of
other potential confounding variables (e.g., verbal intelligence and general academic
achievement scores). Such a result would suggest that both domain-general cognitive
developmental processes and domain-specific mathematics knowledge contribute to the
variability in DPR skills.
A second goal for the first line of research is to explore the nature of DPR skills more
completely. It was claimed that DPR skills are related to but distinguishable from metacognitive
ones. To test this claim, the research will analyze whether DPR skills can be predicted from
measures of students’ general cognitive abilities, metacognitive capacities, executive
functioning, and dispositional learning style, independently of their age, expertise, verbal
intelligence and academic achievement scores. DPR skills are hypothesized to be related
Amsel
8
specifically to measures of EF, metacognition, and dispositional learning style, independently of
age, expertise, and verbal IQ. That is, it is predicted that regression analyses will confirm the
theoretical claim of the specific importance of EF and dispositional learning style in accounting
for DPR skills.
A third goal of the first study is to assess the relation between DPR skills and students’
general academic and specific mathematics achievements. The role of DPR skills in contributing
to academic success generally and mathematics learning and achievement in particular will be
assessed. Students’ GPA, standardized achievement tests (Stanford Achievement tests, ACT)
will be collected as will their performance in math classes in which they are currently and have
previously enrolled (collected from archival sources). Analyses will test whether DPR skills are
related to various academic outcomes independently of age and expertise.
A.1 Participants
The design will be based on Amsel et al. (2005c) and will compare university-level prealgebra and calculus mathematics students to high-school students who are enrolled in the same
math courses. The high school students will come from the same European-American suburban
area as students attending the university, with the former institutions serving as feeders for the
latter. Each institution has been invested in the research, so the research will be an academic
requirement for students, allowing for a potentially very large sample. Amsel et al.’s (2005c)
sample included 110 middle school pre-algebra students, 110 college pre-algebra students, and
74 college statistics students, whose performance on the DRP task (RB-J and RB-E) was
collected on a single day! Although there is no expectation that such a data collection process
would occur for the research covered by the grant, there is the expectation that a sample size of
100+ participants for each condition (N=400) can easily be collected for each study. Such a
sample size has a power rating of 1.0 for a 2 (Age) by 2 (Expertise) ANOVA (Length, 2005;
Friendly, 2005), as it was for the group effects on DPR skills in Amsel et al. (2005c).
A.2 Tasks
DPR Skills. Participants’ DPR skills will be assessed using the Ratio Bias (RB-J and RBE) tasks which vary gambling ratios (1/10 vs. 10/100; 9/10 vs. 90/100; 5/10 vs. 50/100; 1/12 vs.
12/144). Despite their equalities, these ratios are designed to tap different intuitions regarding the
advantages or disadvantages of gambles with larger or smaller absolute numbers of winners or
losers. For example, previous research has demonstrated a shift in preferred gambles from the
1/10 vs. 10/100 condition (a preference for the 10/100) to the 9/10 vs. 90/100 (with a preference
for 9/10) (Leavitt, Bates, & Amsel, 2005). Responses on the RB-E task will be analyzed to
identify one of five categories of regulatory skills. The first four categories are consistent,
meaning that participants generate the same response pattern on 3 or 4 RB-E trials (binomial
p’s<.01 - .001 for each response category, based on Amsel et al., 2000c). Competent Regulators
are certain that having no gambling preference is the only analytic-based processing response on
the task. Even though some Competent Regulators may have a preferred gamble, they are able to
regulate their thinking well enough to evaluate that their response was a product of the
experiential-based processing system and different than the product of analytic-based processing
system. In contrast, Flawed Regulators are certain that only a preferred gamble is an analyticbased response on the task. Flawed Regulators rely on their experiential system to generate
responses on the task and appear to have no recognition that an analytic processing of the
Amsel
9
information may produce a different response. For them, the experiential system is the only basis
to generate responses on the task, so they evaluate such responses as analytic. Finally, Conflicted
Regulators are certain that both having no preference between the gambles and having a
preferred gamble are each analytic-based responses, suggesting confusion between the outcome
of the analytic and experiential processing systems. Other Regulators reliably evaluate rational
response options according to response patterns not otherwise categorized. The fifth regulatory
category includes those who do not reliably respond with the same pattern of responses over the
four trails (Inconsistent Regulators). In previous research, over 87% of the participants could be
reliably categorized into one of the four consistent regulatory styles (Amsel et al., 2005b, c).
Intellectual Ability. Participants will complete the revised version of the Shipley
Vocabulary Test (Shipley, 1940). The Shipley Vocabulary test measures vocabulary recognition
in a multiple-choice format and has proven to be a valid (Prokosch, Yeo, & Miller, 2005) and
reliable (alpha=0.87; Shipley, 1946) index of overall intellectual ability.
General Cognitive Ability. Two measures will assess participants’ general cognitive
ability. The Backward Digit Span subtest of the WAIS scale demonstrates high levels of
reliability and validity (Jenson 1999). Participants will be asked to remember lists of numbers
and immediately after presentation of each list, they will be asked to recall the list in reverse
order. The number of items on each list will increase across trials, from two to seven.
The Figural Intersection Test (FIT) (Pascual-Leone & Smith,1969) measures the size of
participants’ central computing space or working memory and has been used widely (c.f., Kwon,
Lawson, Chung, & Kim, 2004 for a bref review). Participants are provided with a set of shapes
on one side of a page (presentation set) and a set of overlapping shapes on the other side (test
set). The test requires subjects to find the common area of intersection in the test set, from the
shapes in the presentation set (most shapes are common). In some items, a misleading irrelevant
shape (not present in the presentation set) is included among the test set. Altogether, there are 36
items on the test. The number of shapes in the presentation set varies from two to eight; the
number in the test set from two to nine. The number of shapes in the test set is equal to the score
given if the item is marked correct. The test is untimed. The FIT is scored by totaling the score
given for each correct answer in the FIT. The highest possible score is 186. This provides
relative data rather than absolute values for working-memory capacity. A Cronbach’s alpha of
.88 was reported by Kwon, Lawson, Chung, and Kim, (2004).
Executive Function: Executive function (EF) will be tested by two measures: the
Wisconsin Card Sorting Task (WCST) and the Stroop Color-Word Test (ST) (see Ozonoff, &
Jenisem, 1999). The WSCT (Grant & Berg, 1948) is a commercially available, standardized
measure of executive function which is thought to measure cognitive flexibility. Participants
receive a deck of cards which can be sorted by color, shape, and number, and asked to match
each response card with one of four stimulus cards. Performance is scored by participants’
frequency of perseverative responses, in which the participant continues to sort by a previously
correct category, despite feedback that it was incorrect (Heaton, Chelune, Talley, Kay, & Curtiss
1993).
Amsel
10
The computer-delivered version of the Stroop Color-Word Test (Stroop, 1935) measures
cognitive inhibition as automatic and prepotent responses (the color word presented on the
computer screen) must be suppressed in order to respond on the basis of the “ink” in which the
word is written (e.g., the word BLUE is printed in green). The data collected will include
latency and verbal errors for participants to respond correctly on the critical condition where the
word and the color are in conflict.
Metacognition: Participants will complete two measures of metacognition. The first is
the Metacognitive Awareness Inventory (MAI; Schraw & Dennison, 1994). The Metacognitive
Awareness Inventory is a 52-item self-report instrument for use by adolescents and adults
(Schraw & Dennison, 1994). The instrument is comprised of two subtests, knowledge of
cognition, which refers to knowledge about self, strategies, and the conditions under which
strategies are most useful, and regulation of cognition, which refers to knowledge about ways to
plan, implement strategies, monitor, correct errors, and evaluate learning (Schraw & Dennison,
1994). The Cronbach’s alpha for the entire instrument reaches .95 (Schraw & Dennison, 1994).
Each item on the instrument is rated on a 5-point Likert scale with 5 being “Always True” and 1
being “Always False”.
The second measure of metacognition is the Reflective Judgment Interview (RJI,
Kitchener, Lynch, Fischer, & Wood, 1993). In the RJI, participants respond to seven interview
questions regarding four ill-structured problems (e.g., chemical additives to foods and building
the pyramids). The interview responses are a basis for categorizing participants’ level of
reflective understanding on a seven-stage ordinal scale. In general, inter-rater reliability has
been moderate to high. Test-retest reliability on four small homogeneous samples over a 3-month
period has ranged from .71 to .83 (reported in Kitchner et al., 1993), and Cronbach’s alpha has
ranged from .62 to .96 (reported in Kitchner et al., 1993)
Dispositional Learning Style: Two measures will be used to assess dispositional learning
style. The first measure is Stanovich and West’s (1997, 1998) Thinking Dispositions
Questionnaire. Each of 24 items is rated on 6-point scale (1 = strongly disagree; 6 = strongly
agree). The 10-item Actively Open-Minded Thinking Subscale (AOT; Cronbach’s alpha = .62)
measures acceptance and tolerance of ambiguity in knowledge, willingness to postpone closure,
and willingness to change beliefs (e.g., "People should always take into consideration evidence
that goes against their beliefs"). The 9 items on the Absolutism/Relativism Subscale (A/R;
Cronbach’s alpha = .69) assesses whether individuals believe in absolute knowledge or, in
contrast, the inherent uncertainty of knowledge (e.g., "Right and wrong never change"). The 5item Dogmatism Subscale (DS; Cronbach’s alpha = .71) assesses tendencies to assert beliefs as
though they are facts. With reverse scoring for the A/R and DS items, Cronbach’s alpha for the
entire scale was .69.
The second measure is Biggs, Kember, and Leung’s (2002) revised, two factor, study
process questionnaire (R-SPQ-2F) which identifies participants’ strategies and motives for deep
and superficial learning. Deep Learning is defined as intrinsic interest in learning and strategies
to maximize meaning and understanding, whereas Superficial Learning is defined as motivated
by a fear of failure with strategies to maximize achievement (grades) and rote learning. The
measure has four subscales including a 10-item Deep Motive scale (DM; Cronbach’s alpha =
Amsel
11
.62) which assesses intrinsic motivation for learning, a 10-item Deep Strategy scale (DM;
Cronbach’s alpha = .63) which assesses focus on strategies promoting meaning and
understanding, a 10-item Superficial Motive scale (DM; Cronbach’s alpha = .72) which assesses
motivation for avoiding failure, and a 10-item Superficial Strategy scale (DM; Cronbach’s alpha
= .57) which assesses strategies to promote rote learning and grade The two Deep Learning
scales combined had a Cronbach’s alpha of .73 and the two Superficial Learning scales
combined was .64. Each item is scaled on a 5-point scale from 1 (never or only rarely true of me)
to 5 (always or almost always true of me).
Academic Achievement: Participants (or their parents) will also be asked for permission
to retrieve their general academic achievement (Grades, GPA, Achievement Tests Scores) and
achievement data in their mathematics courses (grades, standardized test performance).
A.4. Procedure
Participants will be tested in their school by research assistants specifically trained to
deliver the tasks. The DPR tasks and some demographic information will be collected in
participants’ math classes early in the semester. All the other data will be collected at other
times over the semester by project RAs. The RAs will be equipped with laptop computers to
present the tasks to high school students during students’ study hall period and before or after
school and to college students by appointment.
A.5. Analyses
The different goals and hypotheses of the study will require different analyses. ANCOVA
comparisons between groups will test for age and mathematical expertise effects on students’
acquisition of DPR skills, independently of other potential confounding variables (e.g., verbal
intelligence and general academic achievement scores). Partial correlation and stepwise
regression analyses will examine the relations between the DPR skills and measures of EF,
cognition, metacognition and dispositional learning style, independently of age, expertise, and
verbal IQ. It is predicted that regression analyses will reveal the specific importance of EF and
dispositional learning style measures in accounting for DPR skills. While correlated with the
other measures, the general measures of cognition and metacognition will not predict DPR skills
(see Appendix A). The final central analysis will employ ANCOVAs to examine the role of
DPR skills in general academic and mathematics achievement independently of the other
demographic, learning, cognitive, metacognitive and motivational factors. A key finding
confirming the general thrust of the present research would be that, independent of other factors,
there remains additional variance in general academic and specific mathematics achievement
attributed directly to DPR skills.
A.6. Implications
The results will clarify the precise nature of DPR skills by assessing their development,
relation to other cognitive and learning skills and predictive power to account for academic
achievement generally and in mathematics.
Amsel
12
B. Research Line 2. Relating DPR Skills to Students’ Mathematical Cognition, Learning,
and Achievement.
The second line of research will more directly examine the impact of DPR skills
on mathematical cognition, learning, and achievement in a classroom-based correlation study.
The goal of this research is to develop a more complete assessment of changes in the DPR skills
over the course of a mathematics course and whether there are general age-related differences in
this regard. Students from the middle-school, high-school, and college pre-algebra and English
classes will be repeatedly assessed for their DPR skills on the standard and a new DPR task over
the course of the semester or trimester. Additionally students’ estimated and final math grades
and other outcome measures (standardized math placement tests, questionnaires assessing math
attitudes and anxiety) will be collected.
The first goal of the study is to examine how changes in DPR skills are related to
mathematical cognition. The research will utilize a new measure of DPR skills and further
explore the relation between DPR skills and mathematical cognition. The standard DPR task
(RB-J and RB-E) will be used to categorize students into a regulatory category (Flawed,
Conflicted, Competent, and Other). In addition a new DPR task will be used to hone in on
specific DPR challenges facing students. The new task is the ratio-bias multiple choice task (RBMC) which will present students with correct and incorrect analytic and experiential-based
solutions of hypothetical students on the RB-J task. Participants will be asked to identify which
response is correct and which ones are based on the analytic and experiential processing system.
Responses will be analyzed to assess the extent to which students err in recognizing and
distinguishing between analytic- and experiential-based solutions. Such errors will be used to
predict their solutions on the math task and their status on the traditional DPR test.
The second goal of the study is to examine how changes in DPR skills are related to
mathematical learning. Specifically, the study will assess whether students develop DPR skills as
a function of repeated presentation of the task and instruction in mathematics. The repeated
testing of DPR will be analyzed for evidence of changes in such skills. The extent to which the
changes in performance on the DPR measures can be related directly to students’ learning in
their math class will be assessed by comparison of the math students’ DPR performance to the
performance of students who are enrolled in English classes. Of interest is whether students’
change in DPR skills is related not only to repeated presentations of the task and math
instruction, but also to age. Older students, may have greater cognitive prerequisite skills (see
Study 1 above) and be able to refine DPR skills over repeated presentations of the task,
particularly when enrolled in a math class.
The third goal of the study is to examine how changes in DPR skills are related to
mathematical achievement. The final math grades and other outcome measures (standardized
math placement tests, questionnaires assessing math attitudes and anxiety) of students in prealgebra classes will be correlated with their regulatory skills at each of three assessment periods
over the semester/trimester. Additionally, the study will distinguish between those students who
enter their pre-algebra classes with DPR competence and those who never acquire such skills
over the course of the semester/trimester or acquire them earlier or later in the
semester/trimester. Students’ DPR status will be used to predict their mathematics achievement
Amsel
13
and performance on related measures. It is predicted that there will be positive correlations
overall between students’ DPR skill status and their mathematics achievement and attitudes and
negative correlations with math anxiety. Specifically, students who demonstrate DPR skills
earlier in the semester will have higher mathematics achievement, more positive attitudes, and
less anxiety than those who never acquire such skills or acquire them late in the semester.
B.1. Participants
One hundred middle-school, high-school, and college students enrolled in pre-algebra
and English courses will be the participants in the study. The high-school and college students
will be different students but from the same schools as those used in the first study. The middle
school algebra students will be sampled from the same neighborhood as the other students. The
English course will be a basic one comparable to the foundational nature of the pre-algebra
course. A sample size necessary for an effect size of 1 in a 3 (Age Groups) by 2 (Class)
ANCOVA is approximately 50 participants per cell (Length, 2005; Friendly, 2005).
B.2. Tasks
DPR Skills: The regulatory skill assessment includes the different versions of the ratiobias (BB-J and RB-E) tasks used in study 1 and parallel versions of them in the new format (RBMC). The new tasks target students’ skills to recognize and distinguish between experiential and
analytic processing of mathematical information. An example of the new RB-MC task is below:
Imagine that you go to a carnival and play some carnival games. In one game you can win a
$50.00 gift if you pull out a black jelly bean from a jar of jelly beans. There are two similar jars
to choose from and only one jelly bean can be drawn (without peeking) from one or the other jar.
In Jar A there are 10 jelly beans, 1 of which is black and the rest are white. In Jar B, there are
100 jelly beans, 10 of which are black and the rest are white. The chances of selecting a wining
black jelly bean from Jar A are 1/10 (10%), which is mathematically equivalent to the odds of
selecting a winning black jelly bean from Jar B (10/100 or 10%). Although the odds of selecting
a winning black jelly bean are equivalent for the two jars, some people may have a preference as
to which jar they would rather choose from. Do you have a preference for a jar to select a jelly
bean from in order to have a better chance of winning (drawing a black jelly bean)? Circle one
option.
Option A. I have a preference for one jar over the other.
If so, which one (circle one):
Jar A (1/10)
Option B. I have no preference for one jar over the other.
Jar B (10/100)
The participants are then presented with each of the following four responses.
Response 1: Option A: The jars are mathematically equivalent, but one jar offers an
advantage because of their actual number of black or white beans.
Response 2: Option A: The jars are mathematically equivalent, but one jar is a
mathematically better or more adequate sample of a 10% distribution of black beans.
Amsel
14
Response 3: Option B: The jars are mathematically equivalent although they seem
different, because each jar has exactly the same 10% chance of winning.
Response 4: Option B: They are mathematically equivalent because Jar A and Jar B
each offers an advantage to drawing a black bean (Jar A fewer white losing beans and Jar
B has more winning black beans).
Finally, Participants are then asked a series of questions, including identifying the most
mathematically sound response option of the four listed above, the responses which involved
Analytic Justifications (defined for participants as logical, sensible and reasonable analysis of the
situation, whether or not you agree with it) and Experiential Justifications (defined for
participants as an automatic reaction and gut feeling analysis of the situation, whether or not you
agree with it). Participants’ errors of omission can be identified as those which involve failing to
identify both responses 2 and 3 when asked about analytic justifications and both responses
options 1 and 4 when asked about experiential justifications. Similarly, participants’ errors of
commission can be identified as those which involve incorrectly identifying an experiential
response as analytic, an analytic response as experiential, or both. Initial pilot research found
that college students making errors of commission performed more poorly on the task than those
who performed correctly or made errors of omission.
Mathematics Anxiety. The Mathematics Anxiety Rating Scale (MARS, Betz, 1978) is a
10-item scale, with each item scored on a 5-point Likert scale from 1 (“strongly agree”) to 5
(“strongly disagree”). High scores on this scale would be an indication of a high level of
statistics anxiety. Pan & Tang, 2004) reported a reliability coefficient of .96 for the Math
Anxiety Scale.
Mathematics Attitude Scale: A questionnaire containing 39 closed questions will be used
to assess attitudes towards mathematics (MAS, Schoenfeld, 1989). Each item is scored on a 7point Likert scale (from 1 = "strongly agree" to 7 = "strongly disagree"). The questionnaire was
determined to be highly reliable (Cronbach’s alpha = 0.85, Tsao, 2004).
Academic Achievement: Participants (or their parents) will also be asked for permission
to retrieve their general academic achievement (Grades, GPA, Achievement Test Scores) and
achievement data in their mathematics courses (grades, standardized test performance).
Additionally, participants will be asked to report on their math grades at each testing session.
B.3. Procedure
Administration support for distributing questionnaires in math classes will be solicited.
The questionnaires will be distributed at the beginning, middle, and end of the semester or
trimester. Students of the same age and background who are not enrolled in math courses (e.g.,
English) will be used a control group.
B.4. Analyses
Students’ performance on the new DPR task will be coded as errors of omission and
commission. Participants’ omission and commission errors will be analyzed by age and
correlated (independently of age and other potentially confounding variables) with their
Amsel
15
mathematical judgments and performance on the standard DPR task. Replicating pilot data, it is
predicted that commission errors will strongly correlate with mathematical cognition. Age by
time effects in the development of regulatory skills will be assessed in a 3 (Age) by 3 (Time)
repeated measure ANCOVA, with general academic achievement and standardized math scores
as covariates. It is expected that there will be Time effects, suggesting that DPR skills increase
over repeated presentations within a class. Finally, correlations will assess whether the timing of
students’ regulatory competence (available initially, early in the semester, later in the semester,
or never) is related to overall class achievement and related measures. A 3 (age) by 4 (Timing of
Regulatory Competence) ANOVA will assess the impact of the timing of DPR competence on
math performance. We expect that this line of research will show the extent to which poor
regulators fail to adequately recognize or privilege the rational processing of mathematical
information.
2.5. Implications
This line of research will provide critical information regarding the malleability of DPR
skills and their role in math class performance. The information will be valuable in determining
the timing and length of an intervention designed to directly test the impact of DPR training on
mathematics cognition in and achievement of pre-algebra students.
Research Line 3. Training Pre-Algebra Students’ DPR Skills
The third line of research will more directly examine the impact of DPR skills on
mathematical cognition and learning in laboratory-based experimental studies. The studies will
test whether a training program designed to promote younger and older pre-algebra students’
DPR skills affects their performance on near and far immediate (one day later) and delayed (one
month later) transfer tasks.
Students from the pre-algebra classes described above will be randomly assigned to a
DPR (DPR) training program, a math problem tutoring (MPT) training program, or both. In each
training program, participants will receive series of 20 math problems (requiring about 60
minutes to solve) in the format of the RB-MC tasks – that is, showing 4 possible response
options, with two correct and two incorrect response options and two analytically justified and
two experientially justified response options. The DPR training program will request that
students identify the two analytic and the two experiential responses. The MPT training program
will require that students identify the two correct and two incorrect response options. In the
BOTH training program, the training will alternately focus on DPR and MPT training. The
impact of the training programs on math learning will be assessed by performance on a series of
logic, arithmetical, and algebraic tasks which reflect near and far transfer problems. These tasks
will include arithmetic word problems, combinatorial reasoning tasks, probability problems,
isolation of variable tasks, and others. The goal of this research is to assess the impact of DPR
training on pre-algebra students’ math cognition and learning. It is hypothesized that DPT
training either alone or with MPT training will be more effective than MPT training alone. The
hypothesis is based in part on mixed effects of tutoring on college developmental math students’
performance (Boylan, Bliss, & Bonham, 1997; Casazza & Silverman, 1996) and may not be
effective particularly for weaker students (Maxwell, 1990).
Amsel
16
C.1. Participants
Middle-school, high-school, and college students enrolled in pre-algebra courses will be
the participants in the study. The high-school and college students will be different students from
the same schools as used in the study 2. An initial screening with the standard DPR (RB-J and
RB-E) tasks will identify 75 students in each age group who will be randomly assigned to one
training program, the other, or both. The power rating of such a design is .99 (Friendly, 2005).
C.2. Tasks and Procedure.
DPR skills will be initially assessed with the standard ratio bias (RB-J and RB-E) tasks
which are designed to assess regulatory competence. Students who performed sub-optimally on
the task (i.e., not categorized as Competent Regulators) will be randomly assigned to one of three
training conditions (DPR, MPT, or BOTH). The training will be web-based and students will be
able to access it though a secure web portal (WebCT). Students will log in to the portal and run
the program from that site, which will randomly assign them to a training program. The program
will distribute the training tasks, assess student errors in each training program, and provide
feedback when errors are made.
The training program will involve two types of mathematics problems (Ratio and Algebraic
expressions), which have been identified by pre-algebra teachers at each institution as
particularly problematic for students. A particularly effective (in pilot testing) algebra item is
described below:
Johnny earns $7.50 an hour. For every four hours he works, he gets a $2 bonus or a proportion of
it. Johnny is trying to figure out what his income would be if he worked various numbers of
hours. Which equation best represents Johnny's salary?
Option A: W = 7.5T + 2(T/4) (let T stand for time worked and W equal wages earned)
Option B: Wage = $7.50 x hours worked + ($2.00/4)
The two options reflect correct (Option A) and incorrect (Option B) responses. The incorrect
responses came from a good deal of piloting exploring students’ misunderstanding of how to
express concrete problems in an abstract mathematical form. The incorrect response was created
by student informants who, when asked to complete the task, misrepresented the algebraic
formula by not defining variables. Four response options on the task were also generated by pilot
students, who were asked to explain why each response was correct. These responses were
further edited to maximize their experiential or analytic nature. Response 1 is an experientialbased justification for the correct answer, based only on the fact that a concrete value can be
substituted for the variable. It reflects limited appreciation of the adequacy of the equation, only
that it has concrete properties of equations seen in the past. Response 2 is an analytic justification
of the correct answer because it acknowledges that abstract variables are used appropriately to
model the situation. Response 3 is an experiential justification of the incorrect answer as it
identifies as important concrete and irrelevant or inadequate features of the answer. Response 4
is an analytic justification of the incorrect response because it focuses attention on the
appropriate level of analysis (that abstract variables are used to model the situation), even though
it does not recognize that the formulae is inadequate in handling the bonus.
Amsel
17
Response 1: Option A is correct because it allows you to see both a value for the money earned
(W) and the time worked (T) in the equation.
Response 2: Option A is correct because the equation appropriately models the relation between
wages earned as an exclusive function of a variable representing hours worked.
Response 3: Option B is correct because it used actual money values for the hourly wage and
for the amount of the bonus.
Response 4: Option B is correct because the equation appropriately specifies that wages are a
result of earning money for working a certain amount of hours and the bonus added to it.
Depending on the condition to which students are randomly assigned, they will be asked to
distinguish between rational and experiential responses on the tasks (DPR Training), correct and
incorrect responses (MRT Training) or one or the other (BOTH Training). Incorrect responses
on each item on each trial will be corrected by the computer. Language explaining the correct
responses to the particular question will specifically target whether the justifications or the
answers are being targeted. The computer will present trials randomly, and. record each
student’s performance (response and response latency) on each trial. Students will be instructed
to perform as quickly and accurately as possible. Incentives will be provided for quick and
correct performance, which we expect to keep participants focused on the dimensions of the
problem they are to focus upon. The timing of the training will be decided in conjunction with
particular math instructors to coincide with the timing of the presentation of these topics to
students in class.
The impact of such training on mathematical problem-solving performance will be assessed with
a math quiz based on typical class problems. The set will include problems from the two training
topics (Ratios and Algebraic expressions) as the near transfer task and two topics which were not
specifically trained (Fractions and Factoring) as the far transfer task. Both near and far transfer
tests will be given to participants immediately after training (immediate transfer tasks) and after
one month (delay transfer tasks).
C.4 Analyses
Age effects in the effectiveness of training will be analyzed in a 3 (Age Group) by 3
(Training Group) ANVOVA on response latencies and error rates to items during training. The
effectiveness of the training programs will also be analyzed in a 3 (Age Group) by 3 (Training
Group) ANVOVA on near and far immediate and delayed transfer tests. .
C.5. Implications
The study should reveal the efficiency of more direct (DPR), indirect (MRT), or both
direct and indirect (BOTH) training techniques to promote dual process regulatory skills in prealgebra students. The study will be central in deciding on the final design of the curricular
intervention planed for study 4.
D. Research Line 4. DPR Training as a Curricular Intervention
Amsel
18
The correlational (Study 2) and experimental (Study 3) analysis of the trainability of DPR
skills will provide a springboard to the classroom intervention in middle-school, high-school, and
college pre-algebra courses. The intervention is envisioned for the last year of the grant and will
use the results of previous studies to inform the details of the intervention. The timing of the
intervention during the semester will be determined from the results of Study 2, which will
reveal the impact on math class achievement of having acquired DPR skills earlier or later in the
semester. The features of the features of the intervention will be determined from the results of
Study 3, which will reveal the impact of direct and/or indirect training on DPR skills and math
performance.
Although details of the intervention await the results of this work, its general details can
be described. All students in middle-school, high-school, and college pre-algebra courses will be
required to complete a web-based math support program. Students entering the web portal (Web
CT) will be blocked randomly assigned (by class) to the regulatory training program or a
problem control condition. The block randomization procedure will insure that participants from
each class in each institution will be equally distributed in each training condition.
The DPR training program will have those features experimentally proven in the training
described in Study 3 to be effective in promoting DPR skills, The problem control condition will
receive the same problems as those in the DPR training but without the multiple choice options
or the feedback. The critical feature of the intervention design is that both groups will get the
same problem content. The effectiveness of the curricular intervention will be assessed by math
achievement (Math grades, student retention, and tests) and related measures (Math Attitude and
Anxiety scales used in Study 2). It is predicted that students in the DPR Training program will
outperform students in the Problem Control condition, despite the content of both training
conditions being identical.
D.1 Participants
Middle-school, high-school, and college students enrolled in pre-algebra courses whose
teachers permit involvement in the assessment will be the participants in the study. The sample
size is expected to be very large in each institution, providing enough power to detect main and
interaction effects of age groups training.
D.2. Tasks and Procedure
All students in each class will be randomly assigned to one of two training conditions
(Regulatory Training or Problem Control). The training will be web-based and students will be
able to access it though a secure web portal (WebCT) at school or at home. Students will log in
to the portal and run the program from that site, which will randomly assign them to a training
program. The program will distribute the training tasks, assess and provide feedback regarding
student errors in the Regulatory Training condition. The program will involve all four to types of
mathematics problems (Fractions, Algebraic expression, Ratio and Factoring). The timing of the
training will be based on the results of Study 2. The math achievement and related measures will
be ones used in Study 2.
D.4 Analyses
Age effects in the effectiveness of training will be analyzed in a one-way (Age Group)
AVOVA on response latencies and error rates to items during training in the Regulatory Training
Amsel
19
Group. The effectiveness of the training will also be analyzed in a 3 (Age Group) by 2
(Condition) ANOVA on math outcome measures.
D.5. Implications
The study should reveal the efficiency of DPR training as a curriculum intervention,
fulfilling the IES, Cognition and Student Learning competition Goal 2 of developing programs,
practices, and policies that are potentially effective for improving outcomes,
III. Personnel
A. PI
The PI is an Endowed Professor and Chair-elect of the Psychology Department at Weber
State University. Trained as a developmental and educational psychologist, the PI has conducted
research at the intersection of educational and development psychology (see abbreviated vita).
He is co-author of a book on Scientific Reasoning (With Deanna Kuhn and Michael O’Loughlin)
which is widely cited in both literatures and co-edited two books on the Theories of
Development (with K. Ann Renninger) and Symbolic Literacy (with James Byrnes). Recent
publications of the PI includes studies on counterfactual reasoning (Amsel et al., under review)
decision making (Amsel et al., 2005), logical reasoning (Amsel et al., in press), gambling (Amsel
et al., under review) and learning in children’s museums (Amsel & Goodwin, 2004). He is on
the editorial board of New Ideas in Psychology and is vice president of the Jean Piaget Society.
He has also served as an ad hoc reviewer for over 20 journals.
The PI also has considerable experience with running studies of the sort described in the
research grant proposal. He has run the series of preliminary studies for the grant some of which
some are in preparation or under review for publication. The PI has a long history of working
effectively with and training undergraduates. There have been over 7 formal conference
presentations by the PI or his students over the past year on the topic, involving well over 1000
participants. In all recent publications, undergraduate students served as junior authors or RAs
acknowledged in the publication. PI is experienced with a wide range of statistical techniques; in
particular, those techniques discussed in the proposal.
The role of the PI on the grant will be fourfold: (a) supervising undergraduate research
assistants (esp. data collection, coding, and entry); (b) data analysis; (c) reporting the results in
journals and conferences, CASL, and to the school board, principal, and teachers of the high
school and (d) interpreting the data in support of the designing procedures for the Intervention
study.
B. Co-PI
The co-PI has expertise in science education, specifically in the area of conceptual
change theory, science misconceptions, and students’ understandings of the nature of science.
This background has facilitated recent research on how students define and perceive learning in
various contexts, ranging from undergraduate research experiences to lower division general
education coursework in the sciences.
The co-PI works closely with the training of teachers, both preservice and inservice,
through courses in the College of Science and through outreach facilitated by his work in Weber
Amsel
20
State University’s Center for Science and Mathematics Education. He teaches a course in
developmental mathematics and serves as the co-chair of a university committee charged with
evaluating the developmental mathematics program at the university. His service and
professional position requires that he works closely with teachers and students in local school
districts and in the university. The Pi and Co-PI have coauthored a successful internal grant on
the issues addressed in this grant.
The role of the co-PI on the grant will be two twofold: (a) establishing/maintaining
relationships among school systems, teachers, students, and the research team; (b) coordinating
team activities in the school system and securing permissions to run the work; and (c) working
with the PI developing training program material.
C. RAs
The 4 undergraduate students who will be hired for 20 hours each will have several roles
as well: (a) collect the data; (b) code and enter the data; (c) assist in material production (e.g.,
copying) and in the development of packets for participants. Together with the PI, and the co-PI,
the RAs will be responsible for helping collate materials and data collection.
IV. Resources
The PI’s office and lab are located in the Dept. of Psychology, at WSU – Ogden Campus.
The lab is equipped at present with 10 networked IBM computers used for preparing stimulus
materials, analyzing data, and developing graphics materials. These computers have been
recently been used to conduct and analyze some of the pilot research. The grant will be used to
update some of the computer software so to present the range of tasks. The lab will be used
primarily for running college students, meeting with the research team, record storage, collating
packets and training. Each computer has the most recent version of SPSS for Windows. An
additional room in the lab is used for data storage. The labs are connected wirelessly to a campus
network that provides access to extensive statistical software and Internet resources. The Dept. of
Psychology maintains clerical services throughout the year. The Psychology Department has upto-date copying facilities to which the PI and co-PI has ready access.