Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
WHEN CHANGE DOES NOT MEAN PROGRESS:
HISTORICAL THINKING, INTERTEXTUAL READING, AND DOMAIN-SPECIFIC
EPISTEMIC BELIEFS IN THE CONTEXT OF ONE HIGH-SCHOOL HISTORY CLASS
Liliana Maggioni, Emily Fox, and Patricia A. Alexander
University of Maryland
Paper to be presented at the Annual Meeting of the American Educational Research
Association, Denver (May, 2010).
1
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
2
Abstract
Using think-aloud protocols, field observations, and a variety of quantitative and qualitative
measures, this study investigated how high-school students’ reading of multiple texts, their
ability to think historically, and their domain-specific epistemic beliefs changed over time. It
also sought to identify what pedagogical practices in their history classroom may have supported
such changes. Results from four student informants and their teacher indicated the consolidation
or emergence of reading behaviors, conceptualization of text, and epistemic beliefs that inhibited
building of understanding out of multiple texts. The data also pointed to the ambiguous role that
the use and analysis of primary sources and the activation of prior knowledge may play in the
development of intertextual understanding and historical thinking.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
3
Variety and breadth characterize the information currently made available by the
development and spread of technology. One aspect of this informational space is the availability
of multiple sources addressing a particular topic or question, often in the form of written texts.
While access to a diversity of sources once tended to typify the informational environment of
experts, this multiplicity is now available to an increasing number of people. Thus, investigating
how individuals relate to multiple informational sources has become a key element for
comprehending the ecology in which they learn and live. The building of understanding out of
multiple sources is also linked to core characteristics of well established academic domains
(Rouet, Britt, Mason, & Perfetti, 1996; Shanahan, 2009). In particular, several aspects that are
central to that critical literacy necessary to gain understanding within information-rich contexts
are also essential aspects of reading expertise in history (VanSledright, 2004). The increasing
emphasis on multiple texts in history classrooms thus provides a particularly appealing locus for
investigating how individuals learn to approach a complex informational environment.
Research on reading multiple texts suggests that epistemic beliefs about knowledge and
knowing play a particularly important role in readers’ behaviors, in their standards for
understanding, and in the level of understanding they are able to achieve (Bråten, 2008; Hofer,
2004; Muis, 2007; Ryan, 1984). Similarly, the literature on historical thinking documents that
particular epistemological ideas regarding the nature of historical knowledge and how its
knowledge claims may be justified are also presupposed in the understanding of concepts central
to the development of historical knowledge (Hynd, Holschuh, & Hubbard, 2004; Lee & Shemilt,
2003; VanSledright, 2002a; Wineburg, 2001a). Hence, an interdisciplinary approach that
simultaneously considers epistemic beliefs, single text and intertextual reading behaviors, and
historical thinking seems a promising avenue for investigating this complex environment.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
4
Theoretical Framework
Research in the development of historical thinking across the ages from seven to fourteen
has identified a progression of ideas about concepts such as evidence, causation, empathy, and
historical accounts (Lee & Ashby, 2000; Lee, Dickinson, & Ashby, 1997; Lee & Shemilt, 2003).
Students tended to move from a view of history as isomorphic with the past, to conceiving
historical accounts as the result of the historian’s interrogation of the sources, guided by specific
questions and mindful of the historical context. In between, students had to reckon with the
limited availability of remnants of the past and with the role of human witnesses, thus raising
issues of bias and loss of information.
Studies of pedagogical practices aiming at fostering student ability to think historically
have underscored the importance of discussing the interpretative nature of history and the nature
of historical accounts, developing linguistic tools to address epistemic understanding, and
educating students to the use of disciplinary practices such as sourcing and corroborating
historical texts (Bain, 2000; 2005; Husband, Kitson, & Pendry, 2003; VanSledright, 2002a).
These studies suggest a pedagogical approach in which the processes of learning history and
learning to think historically go hand in hand, an approach that aligns with findings from studies
of student epistemic beliefs, which indicate that epistemic change requires a holistic approach,
sustained over time (Elby, 2001), and explicit discussion of epistemic issues (Brickhouse,
Dagher, Shipman, & Letts, 2002; Ryder, Leach, & Driver, 1999).
Reading of multiple texts involves, over and above the behaviors deployed in reading
individual texts, the ability to manage linking between texts, identification of important
information within texts, and appropriation of evaluative criteria to guide selection of
information or evidence, all in service of building an overall model of the phenomenon or topic
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
5
at issue via an intertext model of the documents and their relations (Afflerbach & Cho, 2009;
Perfetti, Rouet, & Britt, 1999). Little research has addressed how adolescents develop facility in
this complex reading task; among aspects that seem to be problematic or that benefit from
instructional attention are basic reading comprehension (VanSledright, 2002b); awareness of an
author (Paxton, 2002) and of conflict between authors (Wolfe & Goldman, 2005); and control of
the focus of linkages made within and between texts (Bråten & Strømsø, 2003; Hartman, 1995;
Paxton, 2002; Strømsø & Bråten, 2002; Strømsø, Bråten, & Samuelstuen, 2003) and with prior
knowledge (Wolfe & Goldman, 2005).
Further, it is not merely a matter of what the reader does, but of how the reader does it.
The quality of a reader’s execution of strategies and reading behaviors appears to be related to
success in building understanding for both single text and multiple text situations (Fox, 2009).
For example, the quality of connections with prior knowledge appears to be important for
learning from informational texts; such connections can be relevant or irrelevant, and the
knowledge retrieved can be correct or activate misconceptions that interfere with comprehension
and learning (e.g., Kendeou & van den Broek, 2005; Pritchard, 1990; Wolfe & Goldman, 2005).
Similarly, restatements or interpretations can be accurate or inaccurate, elaborations can
successfully expand upon or digress from text content, and evaluations can be principled or
superficial.
Building on a prior investigation of adolescents’ reading of multiple history texts
(Maggioni, Fox, & Alexander, 2009), this study seeks to identify elements of change in students’
historical thinking, reading behaviors, and domain-specific epistemic beliefs during the second
quarter of the fall semester and to suggest the role played by the ecology of the classroom. We
addressed the following research questions:
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
6
1. What patterns of change emerged in students’ reading of multiple history texts, their
ability to think historically, and their domain-specific epistemic beliefs?
2. What aspects of the classroom environment may have played a role in fostering such
changes?
Method
Participants
Four high-school juniors (two female; two males) and the teacher of their honors US
History course participated in this study. Participation of this class was solicited because it is
part of a school system that promotes the use of a variety of primary sources in teaching history
and encourages writing in history. These strategies have been used to foster epistemic
development and the literature has reported some success (Bain, 2000; VanSledright, 2002).
With the help of their teacher, the first author selected four students representing various levels
of academic achievement and attitudes to act as student informants, out of the available pool of
students who obtained parental consent.
Data Sources
Constructed Response Task (CRT). In the CRT, students read a set of 6 written
documents with the purpose of answering a specific question. Two document sets were
assembled, parallel in length, difficulty, and potential construction of argument, with parallel
associated questions. The first, administered in the first session at the middle of the semester,
regarded the landing of Captain Cook on Hawaii; the other, administered at the end of the
semester, addressed ideas about Earth’s shape during Columbus’s time.
Beliefs about History Questionnaire (BHQ). This 22-item, 6-point Likert scale
questionnaire addresses history-specific epistemic beliefs and is a refinement of a measure whose
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
7
factor structure was investigated in previous studies (Maggioni, Alexander, & VanSledright,
2004; Maggioni, VanSledright, & Alexander, 2009). The statements exemplify beliefs
characterizing three theoretically justified epistemic positions, compatible with the
developmental progressions identified both in the epistemic literature (King & Kitchener, 2002;
Kuhn & Weinstock, 2002) and in studies addressing the development of historical thinking (Lee
& Shemilt, 2003; Wineburg, 2001a). Specifically, the developmental trajectory is characterized
by an increased integration of the objective and subjective aspects of knowledge. Statements in
the BHQ reflect such a trajectory by referring to beliefs typical of the following epistemic
stances: a) an authorless view of history as isomorphic to the past (Copier); b) a fundamentally
subjective view of history (Subjectivist); and c) awareness that historical interpretive narratives
result from interaction between historians’ questions and the archive (Criterialist).
Additional questionnaires. Students provided demographic and academic information,
and rated their confidence about learning history during the semester. They completed
questionnaires targeting student need for cognition (Cacioppo, Petty, Feinstein, & Jarvis, 1996),
goal orientation (Harackiewicz, Barron, Tauer, Carter, & Elliot, 2000; Midgley et al., 1998),
intrinsic motivation (Gottfried, 1985), and competence beliefs and subjective task values
(Wigfield et al., 1997).
The teacher completed an open-ended questionnaire about her educational background in
history, professional experience, educational goals, and level of confidence in reaching these
goals in that particular setting, and a questionnaire assessing general interest in history or
participation in professional discourse. A slightly modified version of the interest questionnaire
was completed also by the students.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
8
Evaluation of student essays. This task was adapted from a similar measure used to
understand the knowledge of history teachers (Wineburg & Wilson, 2001). The teacher thought
aloud while evaluating four essays written by students who did not belong to the class; a semistructured interview regarding her evaluative criteria, potential feedback to students, and
recommendations for future instruction followed.
Field notes. The first author conducted eighteen classroom observations across the entire
semester, took detailed notes, and collected copies of instructional materials distributed in class.
Procedure
Students were interviewed at the middle and at the end of the semester. Participants
practiced thinking-aloud until they were comfortable. They were told that the interviewer was
interested in what went through their minds while they were reading the texts and how they built
a response to the CRT question. Students then read the texts and offered a response to the CRT
question, all while thinking aloud, and with occasional prompting or probing for explanations by
the interviewer, in addition to further questioning by the interviewer after the students had
offered their initial response. A structured interview followed, during which participants were
asked to express and explain their degree of agreement or disagreement with the items of the
BHQ. The entire session was audiotaped and later transcribed. Both structured interviews were
based on the statements of the BHQ, while the CRT tasks were similar in format but different in
content at the two administrations.
The teacher’s measures were completed during three meetings at the end of the semester.
A semi-structured interview about broad questions such as “What is history for you?” was
followed by the Student Essay Evaluation Task, the think-aloud for the second student CRT on
Columbus and the structured interview based on the BHQ.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
9
Analysis
Coding of think-alouds and of structured interviews. The literature and prior studies
provided analytical tools to identify facets of intertextual and single text reading, historical
thinking, and history-specific epistemic beliefs, suggested a series of hypotheses about processes
and relations we expected to observe, and provided an initial focus in searching the data for
patterns and topics (Afflerbach & Cho, 2009; Lee & Shemilt, 2003; Maggioni, VanSledright, &
Alexander, 2009; Pressley & Afflerbach, 1995; Rouet, Marron, Perfetti, & Favart, 1998;
Wineburg, 2001a). Appendixes A and B contain the final rubrics for the coding of historical
thinking and epistemic beliefs that emerged from the ensuing iterative process of analysis. A
detailed description of these categories has been included in the reporting of prior studies
(Maggioni, Fox, & Alexander, 2009). Here, we revisit only those categories reflecting ideas and
behaviors that we did not expect on the basis of our understanding of prior literature but that we
created as a response to what emerged from the analysis of the data. In particular, we briefly
consider two categories regarding epistemic beliefs and one category regarding historical
thinking.
Analysis of the structured interviews showed that several participants seemed to oscillate
between believing that historical knowledge is directly produced by the remnants of the past and
believing that it is generated on the basis of arbitrary preferences of the knower. We named
these categories Transition 1(TR1) and Transition 2 (TR2) to capture the state of flux between
apparently contradictory views that seems to characterize this kind of beliefs. In particular,
utterances coded as TR1 expressed the desire that history coincide with the past (a view shared
by the copier stance) while stating that history is hopelessly subjective whenever the remnants of
the past are scarce or debatable (a view echoing the subjectivist stance). Historical knowledge
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
10
was often described as made up of facts and opinions, with the former completely independent of
the knower and the latter exclusively dependent on individual minds. Utterances coded as TR2
reflected awareness of the interpretive nature of historical knowledge (typical of the criterialist
stance) while lacking clarity about the criteria that make such interpretation possible and justified
(a feature shared by the subjectivist stance).
Analysis of the think-alouds showed that participants used strategies or stated ideas
clearly incompatible with historical thinking. We coded these practices as Historical Thinking
No (HTno) to indicate that participants engaged in a process (e.g., skipping the lines containing
the source of the documents because they thought that this information was useless for
accomplishing the task) or affirmed knowledge (e.g., the historical method is not necessary since
one can know history well even without it) that prevented historical thinking. In other words, we
did not code as HTno the mere lack of use of heuristics that would signal historical thinking
(e.g., sourcing); rather we reserved this code to identify the emergence of deliberate practices
and the statement of ideas that make it impossible.
The authors reached a common interpretation of the data from the first administration of
the measures by comparing their independent codings for historical thinking and epistemic
beliefs and resolving disagreements by discussion. Interrater reliability was high (Maggioni,
Fox, & Alexander, 2009), and the first author coded the remainder of the data using these
rubrics. The single text and intertextual reading behaviors produced spontaneously during the
CRTs by the participants (not in response to a specific prompt from the interviewer) were coded
independently by the second author, using sets of codes used previously for the data from the
first CRT (Maggioni, Fox, & Alexander, 2009) and derived from the literature (Afflerbach &
Cho, 2009; Pressley & Afflerbach, 1995). A listing and explanation of these codes is given in
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
11
Appendix C. Consensus on the reading behavior codings was reached by joint review by the
first two authors, in which the initial codings were substantially validated, with changes to only 3
of 186 total single text codes and 3 of 61 total intertextual codes assigned.
Other analyses. The first author analyzed field notes looking for pedagogical practices,
use of primary sources, and kind of tasks assigned to students in class. Artifacts collected in
class were used to aid interpretation of the dynamics observed and analyzed within the context of
the specific class in which they were used. The evaluation of students’ essays was used to
corroborate the teacher’s stated goals for that class. The teacher’s answers to Grand Tour
questions and responses to questionnaires were used to build a richer portrait of her goals and to
integrate responses to the structured interview, whenever history-specific epistemic beliefs were
addressed. Information from students’ responses to written questionnaires was also used to build
a richer portraits of these participants.
Results and Conclusions
The Students: General Trends
Table 1 summarizes changes in epistemic beliefs for the students across the two
administrations. Table 2 does the same for historical thinking, and Table 3 concerns reading
behaviors. In order to compare results, for each student, we transformed the frequency of each
category into a percentage, calculated as a ratio between the frequency of that category and the
total number of codes attributed to her or him in a specific administration of the CRT (for
historical thinking and reading behaviors) or BHQ (for epistemic beliefs). We based the
calculation of the averages reported in Tables 1 and 2 on these percentages. Reading behavior
was treated differently, due to the number of reading code categories used; the data reported in
Table 3 are for the most prevalent single text and intertextual behaviors seen in the two think-
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
12
alouds. No averages are given, as the categories differ across students. A complete breakdown
of the coding for each student’s spontaneous reading behaviors during the CRTs is given in
Appendix D.
Epistemic beliefs. From an epistemic standpoint, participants became generally more
aware of the presence of the knower in the generation of historical knowledge and, in the
aggregate, a higher percentage of student utterances demonstrated awareness of the existence of
criteria that can aid such process. As Table 1 shows, Copier and Transition 1 utterances
continued to comprise most student utterances across the two administrations, although the share
of utterances coded as Subjectivist and Transition 2 increased during the second interview.
Table 1
Frequencies, Percentages* and Averages* of Utterances Expressing Student Epistemic Beliefs
Name
EBCO
Freq. (%)
TR1
Freq. (%)
EBSUB
Freq. (%)
TR2
Freq. (%)
EBCR
Freq. (%)
EB
Freq.(%)
Ashley.1
Ashley.2
9 (47%)
5 (25%)
4 (21%)
11 (55%)
1 (5%)
3 (15%)
5 (26%)
1 (5%)
-
-
Elizabeth.1
Elizabeth.2
8 (53%)
21 (70%)
6 (40%)
9 (30%)
1 (7%)
-
-
-
-
Jack.1
Jack.2
9 (50%)
1 (5%)
4 (22%)
5 (26%)
1 (6%)
5 (26%)
4 (22%)
8 (42%)
-
-
Mark.1
Mark.2
2 (11%)
1 (5%)
5 (26%)
1 (5%)
1 (5%)
1 (5%)
5 (26%)
11 (58%)
2 (11%)
5 (26%)
5 (26%)
38%
28%
22%
34%
6%
12%
13%
18%
15%
3%
7%
7%
Average.1
Average.2
Note. When used after a name, 1 denotes data referring to the first administration and .2 denotes
data referring to the second administration.
*Percentages and averages are rounded to the closest integer; thus the sum by row does not
necessarily equal 100.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
13
Yet, at the same time, utterances suggesting a Criterialist stance diminished, suggesting a more
complex view of the change that took place across student participants. In sum, the direction and
level of change was not consistent across these four students; overall, there was a greater
acknowledgment of the role of the knower, but the integration between object and subject of
knowledge, in a few cases, became more problematic. Further, while Elizabeth and Ashley
showed moderate changes, Jack and Mark demonstrated greater shifts in their thinking.
Historical thinking. The average data reported in Table 2 suggest that overall, changes
in historical thinking were very modest. In particular, evidence of use of heuristics signaling
historical thinking (HTyes; see Appendix A for a description and examples) remained quite
stable across the two administrations, while the use of heuristics hindering historical thinking
modestly increased. We compared overall student performance on the two CRT tasks in order to
check whether differences present at the level of individual students disappeared in aggregating
the data. When considered holistically, we found that students did not differ much in terms of
their ability to think historically across the two administrations. Moreover, although students
differed in their abilities to understand single texts and in the level of interest in the tasks, their
performances in terms of historical thinking remained fundamentally similar. Differences across
the two administrations mainly regarded an increased role of student prior knowledge in building
understanding and constructing the response and a more widespread use of what we called the
“majority rule.” The analysis of reading behaviors confirmed this finding.
Students showed more familiarity (or thought themselves to be more familiar) with the
topic and the period addressed in the second CRT. Specifically, they referred to the belief that
people at the time of Columbus thought that the earth was flat, sometimes adding negative
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
14
connotations of the “Dark Ages,” the feudal system, and the role of the Church. Students took
these pieces of prior knowledge for granted and never questioned them in light of the documents.
In building their responses, students increasingly tended to rely on counting how many
documents suggested a similar view (i.e., people at Columbus’s time believed that the earth was
flat). From these results, they inferred what might have been the prevalent beliefs at Columbus’s
time (if they thought that one view was clearly preponderant among the documents) or how
different beliefs might have been distributed among the population (if they thought that different
views were equally represented in the texts).
Table 2
Frequencies, Percentages* and Averages* of Utterances Expressing Student Historical Thinking
Name
AQ
Freq. (%)
AA
Freq. (%)
HTno
Freq. (%)
CP
Freq. (%)
HTyes
Freq. (%)
Ashley.1
Ashley.2
1 (11%)
1 (14%)
-
5 (56%)
6 (86%)
2 (22%)
-
1 (11%)
-
Elizabeth.1
Elizabeth.2
4 (33%)
9 (32%)
1 (9%)
7 (25%)
4 (33%)
8 (29%)
3 (25%)
2 (7%)
2 (7%)
Jack.1
Jack.2
6 (60%)
4 (25%)
3 (19%)
1 (10%)
-
2 (20%)
5 (31%)
1 (10%)
4 (25%)
Mark.1
Mark.2
3 (8%)
9 (35%)
14 (40%)
5 (19%)
1 (3%)
1 (4%)
2 (8%)
17 (49%)
9 (35%)
28%
27%
19%
16%
26%
30%
17%
12%
18%
17%
Average.1
Average.2
Note. When used after a name,1 denotes data referring to the first administration and .2 denotes
data referring to the second administration.
*Percentages and averages are rounded to the closest integer; thus the sum by row does not
necessarily equal to 100.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
15
In either case, they took the claim of the text at face value and did not corroborate it with other
documents, as if a text’s claim could be considered evidence in itself and a majority of texts
supporting a similar view could be treated as preponderance of evidence.
Reading behaviors. Across the two tasks, comparison of the frequencies of single text
and intertextual codes in Table 3 makes it evident that students in general produced relatively
little in the way of spontaneous intertextual behaviors. In addition, these behaviors generally
involved low-level management of the task demands of dealing with multiple texts, as Table 3
shows. The differences regarding the role of prior knowledge and the greater use of a majority
rule strategy noted above between the responses to the two CRTs in terms of historical thinking
were reflected in the coding of students’ single text and intertextual reading behaviors. For the
second passage, students seemed to identify more readily the poles of the controversy involved,
often with the assistance of prior knowledge. They devoted the bulk of their intertextual efforts
to sizing up each text in terms of which side its ‘vote’ could be recorded on, supported by (often
inaccurate) restatements and interpretations of the content of the individual texts that are seen in
the codes for single text behaviors. At the same time, however, all of the students seemed to find
the second CRT more difficult in some respect, either with regard to the question (Ashley) or
with regard to the amount of effort required to understand the individual documents (Elizabeth,
Mark, and Jack).
Beyond what came out of spontaneous comments during the think-aloud proper (which
supplied the data for the summaries presented in Table 3), probing by the interviewer following
students’ construction of a response to the question elicited further statements often revealing
less functional or facilitative aspects of the intertextual strategies or behaviors the students had
enacted or would use in similar situations. At time 1, the interviewer asked each student about
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
16
his or her failure to take note of the sources, and the responses revealed that students shared a
belief that source information was not particularly useful for interpretation and evaluation of text
contents.
Table 3
Dominant Single Text and Intertextual Reading Behaviors
Name
Single Text Behaviors
Freq (% of total single text codes)
Ashley.1
Question
2 (67%)
Ashley.2
Interpret,
elaborate
3 (33%)
Interpret.
elaborate
1 (33%)
Question
2 (22%)
Elizabeth.1
Restate
4 (36%)
Interpret,
elaborate
3 (27%)
Restate
8 (26%)
Elizabeth.2
Interpret,
elaborate
13 (42%)
Jack.1
Restate
16 (64%)
Interpret,
elaborate
6 (24%)
Jack.2
Restate
9 (45%)
Interpret,
elaborate
6 (30%)
Mark.1
Question
13 (46%)
Mark.2
Interpret,
elaborate
25 (42%)
Intertextual Behaviors
Freq (% of total intertextual codes)
Identify theme
1 (100%)
Connect to
knowledge,
experience
2 (22%)
Reread
2 (18%)
Evaluate
contribution
5 (50%)
Evaluate
contribution
6 (30%)
Identify
theme
3 (30%)
Attend to
theme
4 (20%)
Connect to
knowledge,
experience
2 (8%)
Question
2 (10%)
Evaluate
contribution
5 (83%)
Link text
segments
1 (17%)
Evaluate
contribution
2 (50%)
Focus on gist
1 (25%)
Attend to theme
1 (25%)
Interpret,
elaborate
6 (21%)
Argue
3 (11%)
Reread
8 (14%)
Use different
text
characteristics
1 (17%)
Use different
text
characteristics
1 (7%)
Evaluate across
texts
1 (17%)
Restate
8 (14%)
Perceive
diversity,
complementarity
3 (50%)
Focus on gist
3 (21%)
Connect to
knowledge,
experience
2 (6%)
Attend to theme
1 (10%)
Perceive
diversity,
complementarity
3 (15%)
Perceive
diversity.
complementarity
1 (7%)
Note. When used after a name, 1 denotes data referring to the first administration and .2 denotes
data referring to the second administration.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
17
At time 2, the interviewer probed for what students did to address texts that did not all
present the same story. Along with at least face-level acknowledgment of the use of somewhat
more functional strategies and behaviors (such as registering that the texts presented different
points of view), the comments of the students included statements indicating that their
procedures and criteria for dealing with multiple conflicting texts discounted the importance of
perceiving and distinguishing relevant text characteristics and essentially dismissed
considerations of critical evaluation of validity and reliability of texts in relation to an overall
representation of the entire set of documents. Instead, they generally proceeded by way of a
more or less superficial text-by-text capture of what was taken as gist information supporting the
identification of the side of the controversy taken by each text, in service of the use of some
version of a “majority rule.”
However, besides these common traits, students’ ideas and patterns of change showed
important differences. Thus, in describing the developing epistemic ideas and approaches to the
reading of multiple history texts that emerged from the data of these students, we now move our
focus to each individual student.
Individual Students
Ashley. Considered as a whole, Ashley’s responses across the two structured interviews
conveyed a similar epistemic stance. While she continued to conceive history as the past (e.g.,
“History is already done and it is never going to change” (Time 2 (T2)), she also thought that it
was accessible through its remnants: “There is documents that we found, dinosaurs bones and
stuff like that […] None of us was there, but we can still [silence] some things we can be sure
of.” (T2). Technology was perceived as a potential ally in the discovery of the past, “because
technology and things that we have are capable of knowing things” (Time 1 (T1)).
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
18
Yet, Ashley also voiced the belief that complete knowledge of the past is always, or at
least very often, impossible because the interpretation of what we have left from the past is
debatable, conflicting, or simply based on too little information. In these cases, she tended to
view history as a hopelessly subjective endeavor that easily became just a matter of opinion,
echoing several of the ideas characterizing a subjectivist stance.
[Y]ou really don’t know history; it’s just through books and people writing down stuff
and documents from back in the days; there could be something missing that nobody
knows about, but […] everybody has a different opinion about history and what they
think happened (T1).
The view of history as isomorphic (at least ideally) with the past and the perception of its
unbounded subjectivity emerged from both interviews; yet, their simultaneous presence in the
same sentence during the second interview occurred much more often in the second interview
and made us code a higher number of utterances as TR1, in the latter case.
In general, Ashley’s performances on both CRTs suggested little awareness of how
historical knowledge can be generated through the analysis of multiple texts. Her understandings
were especially problematic in regard to sourcing, corroboration, evaluation of sources, and
construction of argument. For example, when directly asked about her use of the source of the
texts, Ashley said that she would use this information in two cases: “so they know that I am not
plagiarizing” (T1), and, similarly, if she used a quote from a document and needed to state the
source for that quote. Her response remained consistent across the two CRTs, confirming that
Ashley continued to treat the documents as authorless text.
In general, Ashley found the second CRT more difficult, because the question seemed to
her “more opinionated” and not based on facts. She commented that “it would [have been] a lot
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
19
easier if we had these colonial charts,” showing Columbus’s depiction of the world; an approach
compatible with the idea that historical knowledge is somehow and only contained in some
remnant of the past. Since Ashley could not find a straight answer to the question in the texts,
she dismissed any reference to the documents, thus increasing the instances coded as HTno.
Conversely, the tendency to introduce in her constructed response elements extraneous to
the sources increased. In particular, Ashley seemed to build on several misconceptions about the
Middle Ages and Columbus. For example, she began addressing the question by recalling her
prior knowledge: “The shape of the Earth that they had, wasn’t it flat? […] It wasn’t round or
anything.” Then, she remembered “reading about Columbus” and the fact that “he thought that it
[the earth] was flat,” and that “at the end, [one would] just fall off.” She kept revisiting this idea
during the whole think-aloud, compared it only with Document 2, ignored all the other texts, and
concluded that “they had no way of knowing, unless somebody went round the whole world and
that would take a very, very long time.”
Ashley did not appear to be a strong reader. Her reading behaviors in relation to the
individual texts showed similar limitations across both tasks, with her spontaneous comments
during reading involving questions about word meaning. When it came to answering the
question, for the first CRT she was at least able to identify what she saw as a common theme
across the passages, while for the second she confined her reference to the documents to
interpretations and elaborations of document 2, along with references to her prior knowledge
about Columbus and her own beliefs about the shape of the earth.
Upon probing by the interviewer at Time 1, Ashley gave her criteria for managing
disagreement between documents as reliance on eyewitness accounts where available (although
it was not clear how she thought an account was determined to be that of an eyewitness, as she
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
20
made no use of source information) and lining up similarities and differences across the
accounts. This was reinforced by her comment during the BHQ about dealing with conflicting
evidence: “you can have two documents and then get a general view, so yeah, you can compare
them and see what is different and what is alike about them.” It did not appear that she had any
strategy for dealing with more than two documents, however.
At Time 2, Ashley dismissed the usefulness of actually revisiting the documents to see if
her recollection that Columbus thought the world was flat was supported. She felt that for the
documents to be useful in answering the question, they needed to “hold more facts about
Columbus,” further indicating that her criteria for evaluating the contributions of texts in a
document set did not include consideration of text characteristics, validity, or reliability.
Although she was not able to use her compare and contrast strategy to answer the question, she
repeated that this is her typical behavior in dealing with conflicting evidence in her comments
during the second BHQ interview: “Get two documents, I do compare and contrast, what is
different, what is the same about it.” Ashley was thus still limited to identification of differences
across two documents, and did not appear to be able to handle the demands of comparing more
than two differing accounts by way of a more developed application of a majority rule.
Elizabeth. Across the two interviews, Elizabeth offered several examples of ways of
thinking consistent with the Copier stance; their relative number actually increased during the
second interview. Specifically, Elizabeth voiced the belief that history coincides with the past
(e.g., “The past is history, so…what does inquiry mean?” (T2)), and, in particular, it is seen as
the series of events that happened in the past. As such, history came to be determined by its
remnants, as this quote illustrates:
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
21
[T]he past is what the evidence makes it to be, what evidence you collect; it’s not the
historian. Historians can say anything about the past and it can be wrong, but the
evidence says what happened in the past. (T1)
Consequently, she viewed the historians’ role as “go[ing] out and investigat[ing] about the past
and find[ing] different stuff” (T2).
Yet, at times, she admitted that “you can’t really know if the history is 100% accurate”
(T1), because people could not “go back in time and get the evidence” (T1) and evidence may
conflict. Even in these cases, however, she was confident that it was possible to find enough
evidence to get to the right story, because,
if it really happened the way it was supposed to happen, one evidence will overpower the
other. You may have two things that conflict with each other, but then you will probably
find more evidence that overpowers the other evidence and says that one is wrong and
the other is right. (T2)
Conceptualizing evidence as detached from argument was well in accordance with
Elizabeth’s beliefs that, in the end, stories cannot differ too much, either, because “evidence says
what happened in the past” (T1). Thus, she felt comfortable in acknowledging that “everyone is
going to have a different opinion about what happened, or should have happened, or why it
happened” (T1), and suggested that one “should have to listen to everyone,” as “when you sit in
the class and whomever speak, you are taught not to voice your own opinion but to listen to the
others’ opinions and take it in” (T1).
In one instance only, during the second interview, Elizabeth seemed to realize the
challenge posed by her epistemic stance and entertained the possibility that her approach could
lead to unreliable historical knowledge:
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
22
Actually you don’t know [what basically happened] unless you find someone that was
there and goes: ‘Hey, that really happened!’ But you can’t really, you don’t really know
what happened, and so one thing could really overpower the other and be completely
wrong.
Yet, when the interviewer asked Elizabeth whether she would have liked to be taught
how to deal with conflicting evidence, she laughed, and answered in the negative:
[T]hen you have to think and to be like “Oh, what is, which is right?” And then you can
make the mistake of being wrong and then you’ll be “Oh!” and then you’ll tell everyone
the wrong thing and change what really happened.
In Elizabeth’s view, in order to be good at history, one needs “to remember who people are, stuff
like that, and reading has nothing to do with remembering” (T1). Finally, consistently with the
view of history illustrated by these quotes, Elizabeth acknowledged her unfamiliarity with the
term “historical method,” and voiced skepticism about the very existence of a historical method:
“[S]cience has a method, but history […] just happens, so there is not really a method” (T1),
Elizabeth approached the two CRTs similarly. Consistently with the epistemic stance
emerging from the structured interviews, she seemed to approach these tasks under the
assumption that the answer would emerge directly from the texts. One difference we noticed
across the two administrations regarded Elizabeth’s increased awareness of and willingness to
reckon with the issue that different texts suggested different answers to the proposed question in
the second CRT.
Specifically, while during the first CRT, Elizabeth tended to force the interpretation of
each text to reach consensus (or simply dismissed texts that she could not fit), during the second
CRT she seemed to assign each document to one of two camps (i.e., round vs. flat Earth) and
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
23
came to a conclusion by counting how many texts supported each belief. In other words, she
appeared to abide by a sort of majority rule. For example, after reading all the documents of the
second CRT, Elizabeth began answering the question by saying that “the prevalent belief about
the shape [of the earth] was kind of 50/50 back then.” When asked to explain why she came to
this conclusion, Elizabeth reconsidered all the documents and assigned each one of them to the
“flat” or “round” camp; at this point, she realized that only two documents, in her view,
supported the idea of a flat earth and thus she modified her initial response stating that “most
people believed that the earth was round.”
From the historical thinking point of view, Elizabeth did not show any significant change.
Specifically, corroboration across documents remained minimal and limited to comparing the
answers to the question suggested by the various texts: “So, the first document is saying that it is
not round, the second document that it is round.” In addition, Elizabeth continued to ignore the
source of the documents and approached them as authorless texts, although she often referred to
the content of the texts using personal pronouns (e.g., “they are saying”).
At the end of the first CRT, when explicitly asked about her behavior, Elizabeth
dismissed the possibility that considering the source could aid in the interpretation of the
documents, commenting that the inclusion of the source was “just redundant” because, although
it reported the author of the text, it had nothing “to do with the question.” She seemed to
maintain this same attitude also in the second CRT; for example, when asked to explain what
criteria she followed in evaluating the different documents, Elizabeth said that they seemed
accurate, because “they pretty much gave people, and place, and the time, Document 5 gives like
statistics; I mean, if you have that, it’s pretty accurate when you have stuff like that. I believe
them.”
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
24
In her reading of the documents for the CRT, Elizabeth’s actual reading procedure
differed somewhat across the two administrations. The first time, she read through all of the
documents and then returned to consider how to answer the question, which she did by
selectively restating or interpreting elements from each document to collect bits and pieces
supporting the idea that the Hawaiians viewed Cook as a god. When questioned by the
interviewer, Elizabeth voiced the view that the documents “in a way, say the same thing…they
all say the same thing about him.” She thus resolved the issue of conflicting accounts by
creating an overall view of the document set in which conflict had simply been dismissed
For the second CRT, Elizabeth used a methodical and organized approach; as she read
each document she targeted words giving clues to the position regarding the shape of the earth
taken or supported by the document, on which she based her interpretation or restatement
(generally inaccurate) of the document’s stance on the issue. She could see that the documents
conflicted, and when questioned by the interviewer, confirmed that her way of dealing with the
conflict was to go with whichever position had more documents on its side. Elizabeth was
confident in her reading skills and in her ability to comprehend texts, as she stated in both
administrations of the BHQ, and seemed to be relying on her own ability as a reader to extract
relevant or gist information. In the absence of any evaluative criteria or indeed of any need for
evaluative criteria relating to accuracy, trustworthiness, or relevance, conflict was resolved by
the simple math of comparison of quantities. When prompted to consider the dates of the
documents, Elizabeth allowed that the criterion of nearness in time to the event at issue could be
used, which amounted to taking the older documents as more accurate, boiling down again to a
simple tallying of new versus old.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
25
Jack. In the first interview, Jack showed little evidence of being able to reconcile the
subjective and objective aspects of knowledge. On the one hand, he thought that “history is
about the past, the events that happened in the past, not really anything else.” In order to know
about it, people can use records and, “nowadays history can be made with videos and stuff, and
so know what happened.” On the other hand, he was also aware that historians interpret the
remnants of the past, or, at a minimum, they need to “reconstruct an event based on these
different records.” However, a lack of understanding of the role played by the historian’s
question and of the purpose and features of the historical method made him unable to reconcile
these ideas.
On a couple of occasions, his impasse surfaced during the first interview. The first
instance was in response to the statement “Historical claims cannot be justified because they are
simply a matter of interpretation.” Jack commented: “I somewhat disagree with this, because
historical claims [silence]. I somewhat agree with this because historical claims is pretty much
interpretation by historians [silence] Ah, I don’t know.” The second instance was in response to
the statement “Reasonable accounts can be constructed even in the presence of conflicting
evidence.” Jack responded: “I somewhat agree with this, because the evidence could be from
like, sources could be from different things that could be, accounts could be skewed from
different sources because, sources could be [silence] I cannot answer this question.”
In the second interview, the role of the knower became clearer; at a minimum, one could
“put the documents together and take the most reasonable information.” The difference between
opinion and interpretation also began to take shape:
[I]nterpretation is like figuring out what happened, opinion [silence], interpretation is
taking all the facts and putting together and see what happened. Opinion is what you
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
26
think that happened, like with no facts, just what you would think is happening, but with
no facts.
However, Jack still lacked criteria for building historical understanding and, although he could in
theory conceive that different historians might interpret the evidence differently, he did not have
much experience of conflicting historical accounts: “I read a lot of history books about the same
event and pretty much say the same thing.” Thus, his heightened sense of the role of the knower
made him at times lean toward a Subjectivist position: “If you interpret something, you put facts
together, I guess, for what you think that happened, and other people can have different
interpretations;” “[W]e weren’t there, so we can’t possibly know what happened, even if there is
records, they can interpret them differently and form their own position.”
Although the method for generating historical knowledge remained fundamentally
unclear to him, Jack seemed to think that a workable approach could consist in extracting pieces
of information from different sources and putting them together. For example, Jack
acknowledged that, although eyewitnesses may disagree, historians “can still piece together
something that happened, based on the evidence […] and see what a reasonable story [it] would
be.” He also saw in the reference to a common method a possibility of overcoming disagreement
among historians, since sharing a common approach might prompt them to “come up with the
same information.”
Jack’s performance on the CRTs illustrated what he probably meant by “piecing
together” different kinds of evidence. Jack approached the two CRTs quite similarly. In general,
he paused after reading each document and summarized its contribution to the question asked by
the task. On the whole, his summaries reflected the text relatively faithfully. However, not all
the elements that he identified while focusing on each document eventually contributed to his
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
27
answer. For example, during the first CRT, after reading Document 2, Jack noted that the
document suggested that the priest did not think Cook and his men were gods:
So this priest pretty much knew that they were not gods, but to be sure that they were not
gods he pretended that they were gods. Most of the other people thought that Cook and
his men were gods and that their giant ship was a floating island. This is pretty much it
for this one.
He also noted that, according to document 6, “the European said that they were gods and the
Hawaiians believed them and not that the Hawaiians thought of them as gods,” which he
interpreted as rejecting the idea that the Hawaiians simply mistook Cook for their returning god
Lono. However, immediately afterwards, in answering the question posed by the task, Jack said
that “based on the documents the Hawaiians thought that he was the god Lono and that they saw
him at Makahiki and went with their ships and gave them things from the island.” When
questioned, Jack confirmed that he interpreted document 6 as rejecting this view; however, he
still considered his answer reasonable because, five out of six documents supported it, suggesting
he was applying a sort of majority rule. In the process, his observation about the priest’s doubts
was also dropped.
Overall, Jack approached the documents as authorless texts; when asked explicitly, he
provided the following explanation:
I don’t read the author, I kind of sort of glance at it, so I can pretty much absorb
information, pretty much. I don’t really use the author, as long as it is not in a response
or anything.
This behavior continued also during the second CRT and hampered Jack’s attempt to integrate
the contributions of different documents. In fact, though he seemed more willing to build an
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
28
understanding that kept into consideration different perspectives, he had no criteria to evaluate
and corroborate the texts. Thus, he was powerless when confronted with disagreement among
them; he solved it by placing their content side by side:
Actually, at the time of Columbus, people didn’t really know what the shape of the earth
was. I think most people that were religious, the Church may sound like the earth was
flat, so, if you were a Christian at the time, I guess, you might have believed that the
earth is flat, but if you were a scholar or weren’t religious and believed in science, I
guess, you believed that the earth was round. But I don’t think anyone actually knew
until the proof came.
At the same time, Jack dismissed those texts that he found hard to understand. This is Jack’s
explanation of the process that he followed in order to build his answer:
Document 6, I kind of, just says the whole thing was an error; I kind of reject this
document, all the other documents pointing to other things. And Document 3, I don’t
really understand, so I kind of reject that one. So, the other ones are the ones that I
believe, I guess, yeah, I pretty much stick with that: the Church thought that the world
was flat, scholars thought that the world was round.
In reading the documents, for the first CRT, Jack began by orienting himself with a
connection to prior knowledge, which then played a role in his development of an understanding
of the documents and their contributions in answering the overall question. Jack commented
after reading the first sentence of the first document that he was reminded of Christopher
Columbus, although the document set concerned Captain Cook. Jack’s recollection of what he
knew about the reception of Columbus and the Spaniards by the native peoples became a
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
29
touchstone for validating what he was seeing the current documents as saying by comparison to
what he knew about this other story.
For the second CRT, Jack commented when prompted by the interviewer before
beginning to read that he was expecting the documents to say that people thought the earth was
not round, because that was what he knew about it. In the course of reading the documents, he
became increasingly settled on a view that belief that the world was flat was connected with
religion, and belief that the world was round was connected with science; he confirmed this for
himself as he went through, giving summary interpretations and restating what he read.
Although on the one hand, he was willing to allow that people at the time could have differing
viewpoints (unlike his suppression of the priest’s doubts in his final general statement about what
all Hawaiians believed in the first CRT), his only recourse when faced with conflicting
viewpoints among the documents in the second CRT was to reject those that threatened his
relatively workable interpretation, which corresponded as well to what he already had in the way
of prior knowledge.
Mark. Before analyzing Mark’s change in epistemic beliefs across the two timepoints,
we must note that he may have been more tired the second time he responded to the BHQ
statements. Mark dedicated a much longer time to the completion of the second CRT due to his
decision to read over the set of texts twice. It is thus possible that fatigue played a part in his
responses during the second structured interview. His language tended to be more casual, in
contrast to the first interview when he chose his words very carefully.
In the first interview, Mark clearly differentiated between the past and history. Within
history, he believed that there was “a certain amount of truth that is set in stone, like the events
that happened,” a truth that would stand “whatever point of view you have of an event,” “no
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
30
matter where you come from.” However, the way “to come upon this truth” required “reading
and learning from different interpretations.” Mark reiterated the key role played by
interpretation several times during the first interview. For example, he attributed disagreement
about past events more to a “lack of understanding of different perspectives” than to lack of
evidence and noted that “facts may speak for themselves, but they don’t think for themselves.”
He added that, since students may find it difficult “to understand history simply from facts,”
schools should help them “to synthesize […] the complex ideas that need to be learned to
understand history.” Mark reiterated this idea also during the second interview, explaining that
“there is a way in which an event actually occurred and then there is multiple, different ways that
it is interpreted, but history itself is more of a boiling down of the different ways in which it was
interpreted to find out the truth.”
Mark also demonstrated in the first interview that he had developed several criteria to
accomplish the work of interpretation; these criteria enabled him to differentiate clearly between
opinion and interpretation: “[H]istory is not necessarily basically a matter of opinion; I believe
it’s a matter more of interpretation and gathering from different sources.” When asked to
elaborate on what “skills” students should have in order to learn history well, he volunteered “the
ability to gather information, the difference between fact and fiction, based on the credibility of
evidence.”
While students tended to be helpless in confronting conflicting or biased sources, Mark
observed that “conflicting evidence […] usually leads to the most reasonable account and more
accurate account, because it presents more than one point of view of an event or an idea. It helps
just diminish the bias of a certain event” (T1). He also added that, although first-hand accounts
“obviously include bias from people,” “biased or not it is still evidence” (T1).
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
31
However, during the second interview, Mark seemed to conceive the existence of
perspective more as a necessary evil than as a positive attribute of human knowledge. As such,
the tendency of historical accounts to “color, and change, and make the event appear differently
to others” was perceived as something inherently biased, something to cut through in order to
reach the unadulterated past. His responses to the statement “Good students know that history is
basically a matter of opinion” illustrate this shift.
I don’t necessarily agree with that conclusion, I somewhat disagree. Good history, I
mean history is not necessarily, basically a matter of opinion; I believe it’s a matter more
of interpretation and gathering from different sources […] There are events that
happened and may be more than one perspective; there is, obviously, from each
perspective there can be a certain amount of truth, otherwise there wouldn’t be so many
perspectives. (T1)
I strongly disagree. As I said, I don’t think that history is an opinion, I think history
should be a fact. It’s just, history should be a fact that is based upon, I guess, based upon
the opinions of more than one source, an opinion being an historical account from one
person, because historical accounts obviously can be biased. (T2)
Thus, in the second interview, Mark strongly agreed with being taught to deal with
conflicting evidence because “this would support the idea that history consists of facts that are
gathered from several pieces of evidence.” He also strongly agreed that comparing sources and
understanding author perspective were essential components of the process of learning history
because these heuristics “are entirely to delineate biased and unbiased information.” These
statements seem more in line with the Copier and Transition 1 stance than with the Criterialist
stance conveyed by several of Mark’s statements during the first interview. Although the issue
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
32
of bias emerged also in the context of the first interview, it was only during the second one that
Mark clearly pitted it against the idea of history as facts. Although Mark continued to
acknowledge the role of the knower in the generation of historical knowledge, in the second
interview he seemed to perceive it as just an obstacle (bias) that prevented the possibility of
accessing the truth of the past. The lack of experience with the historical method that Mark
mentioned during both interviews may have played a role in his perception.
We found elements of this epistemic shift reflected in Mark’s performance on the CRTs.
Compared to the other participants, Mark demonstrated a higher frequency of use of heuristics
that may facilitate thinking historically. On a general level, Mark interrupted the reading of the
documents to interject comments and questions much more often than the other participants.
His first think-aloud showed a constant questioning of the writers of documents, during
which Mark often challenged the trustworthiness of the accounts. For example, in reading that
Captain Cook was offered a hog in sacrifice, Mark asked: “How do they know it was in sacrifice
and not just a gift?” Again, after completing the reading of Document 1, he commented:
My thoughts here is that when I read ‘these distinguished civilities were never offered by
the islander to a mere human beings’ […] I question the accuracy of that, because if this
is the first time that they landed on this island, is there really a way that they can know
about that?
In addition, Mark paid close attention to the perspective offered by the documents and
noted differences between texts; he mostly used cues provided by the language employed by the
authors to guess about their point of view and the purpose of the account. For example, after
reading Document 5, he said:
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
33
This is completely contrary to the first document I read, that said that they received him
as a god and took him to the main town. This seems more actual, more didactic, the
other seems more of a fictitious story; it sounds like a grand jury report […] maybe by
Cook to his […] native land to make himself, white people, look…
Yet, like the other students, Mark never looked at the source of the texts while working on the
first CRT, although their examination would have provided an answer to a few of his questions.
Thus, his consideration of perspective fostered a generalized suspicion toward the authors of the
texts more than providing a tool for using the documents available to address the question asked
by the task.
For the first CRT, Mark was also careful in considering the cultural context in which the
events took place. He tried to empathize with the Hawaiians and imagine how the events may
have looked like from their point of view; contrary to the other students, he found Document 4
particularly useful in this respect:
When I read this, I kind of think about how probable it is that the native Hawaiians
received Captain Cook believing that he was a god, because he says that other
Polynesian people did the same. […] [T]his kind of makes me think about the
Eurocentric view, how the native Hawaiians, the Polynesians received them, the kind of
European perspective seems a little arrogant, obviously.
The questioning attitude and willingness to accept the provisionality of his understanding
of each text seen during the first CRT from Mark were less evident during his reading in the
second CRT. He did much more summing up and formulating of interpretations as he went
along. When reading the second document, from St. Augustine, Mark made a lengthy
connection to his own personal experience (through his father) with biblical studies, and also a
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
34
connection to the idea of Pangaea as explaining how there could be people on the other side of
the earth. Although Mark read carefully, commented on the difficulty of understanding several
of the documents, and re-read the entire document set before answering, his orientation toward
the idea of the Church as being biased and holding antiquated ideas resurfaced in his response
and colored his interpretation of the evidence presented.
During the second CRT, Mark said that he had “paid more attention” to the authors of the
sources, trying to find out whether the text was written in someone’s interest or to identify the
perspective from which the document was written. However, for the most part, Mark seemed to
take the documents at their face value, although he expressed the view that some documents
needed a close scrutiny, because of greater bias:
[T]he source from the Church […] just because it was from the Church and during this
time period there was, I know for a fact, a lot of corruption things in the Church and […]
it seems it’s the Catholic Church, which means more corruption at that time period […]
that shows certain bias in that document so that it hasn’t to be taken at face value.
In general, Mark was more critical than his classmates toward the documents he read.
However, once he acknowledged the presence of a historian, he had no criteria for using this
awareness to foster his understanding and evaluation of the sources, other than by way of
consideration of issues of perspective and bias. This lack of appropriate heuristics made Mark
stop shorter than what the documents would have allowed him. Thus, after reading twice all the
documents comprised in the second CRT, he concluded:
It is difficult to answer. It is really hard to say based on these documents what the
prevalent idea was from the people, because a lot of these documents do not really
reference what the people thought, because this is, I mean, a document of the Church –
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
35
Document 2 – Document 1, I guess, is just a book about Columbus, that was published, I
am not sure whether it was a text in someone else’s interest. [silence] I don’t really
think, from the documents, it doesn’t really show that there was a general consensus
about the shape of the earth at the time of Columbus. Really the documents, I think,
conflict too much to, not necessarily conflict, but they paint a picture that does not
necessarily explain what the general consensus was.
Further, the data do not suggest that Mark would have been in a better position to make
up his mind if he had not been constrained by a given set of documents. When asked after the
second CRT how he would have pursued the inquiry further, Mark said that he “would probably
use the internet,” do “a vague search,” and then “take different pieces of information from
different sources.” He also stated that he wouldn’t believe all that he read, but when pressed to
elaborate on the criteria that he would use to make his decisions, he mentioned only how he
would detect issues of bias.
Other Cognitive and Motivational Variables
Table 4 reports results from questionnaires addressing other cognitive and motivational
variables. These results indicate that the selection of participants to obtain representation of a
range of abilities and attitudes was quite successful. In some cases, such as Mark, we believe
that these findings may concur to explain his unique performance and persistence on the tasks.
Compared to other participating students, his need for cognition, interest in history, and mastery
orientation were especially high. In other cases, it is difficult to interpret these findings as
possible explanations of students’ performances. For example, with the exception of the score
on interest in history, Ashley’s scores are consistently higher than Jack’s. Yet, Jack’s trajectory
in terms of historical thinking and intertextual behaviors appeared to be more adaptive that
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
36
Ashley’s. It may be that a measure of reading comprehension ability would also have been
useful in explaining what was seen from these students; for instance, Ashley did not appear to be
a strong reader.
Table 4
Scores on Intrinsic Motivation (IM), Need for Cognition (NC), Competence Beliefs (CB),
Subjective Expectancy Value (SEV), Self-Efficacy (SE), Interest in History (I) and Goal
Orientation.
Name
IM
NC
CB
SEV
SE
I
MA
PAp
PAv
WA
MAX
value
90
108
30
36
10
117
36
36
30
30
Ashley
70
74
22
27
6
32
30
14
8
10
Elizabeth
51
30
12
13
5
24
25
27
24
20
Jack
50
70
19.2
21.5
7
58
24
16
12
15
Mark
65
90
23.5
16.8
8
60
33
21
8
14
*Goal Orientations: Mastery (MA), Performance Approach (PAp), Performance Avoidance
(PAv), Work Avoidance (WA)
First Research Question: Summary of Findings
What patterns of change emerged during this period of time in students’ reading of
multiple history texts, their ability to think historically, and their domain-specific epistemic
beliefs? Although these four students expressed different epistemic ideas when we met them for
the first time, results suggest that all of them tended to become more sensitive to the presence of
a knower in the generation of historical knowledge, when we interviewed them at time 2. At the
same time, the lack of criteria that can aid this process persisted. Thus, students related the
presence of a knower almost exclusively to the presence of bias. We found them often
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
37
oscillating between the belief that historical knowledge is safely conveyed by the remnants of the
past and the unsettling realization that this might actually not be the case, once they reckoned
with the role of interpretation. Yet, students also differed in how they faced this problem.
Ashley sharpened the dichotomy between facts and opinions, while Elizabeth declined the
epistemic responsibility of facing the issue. Jack tried to work his way out of this epistemic dead
end by trying to piece together different kind of evidence, but the issue of interpretation
remained unresolved. Yet, we found it remarkable that he hinted at the usefulness of a common
method to reach some agreement among different interpreters. Similarly to Ashley, Mark also
sharpened the difference between historical facts and historical opinions and came to see the
issue of bias as an evil to be overcome.
The lack of criteria (e.g., sourcing, corroboration, and contextualization) to build
historical knowledge and of effective intertextual reading strategies remained evident across the
two performances on the CRT tasks. Prior knowledge seemed to play an even greater (and
negative) role during the second CRT. Beside these commonalities, students’ differences in
performance varied. Ashley went from considering the contributions of different texts to a
common theme (T1) to focusing mainly on one text (T2), demonstrating the weakness of relying
merely on a compare and contrast strategy. Elizabeth went from forcing a single interpretation
on all the texts (T1) to the use of the “majority rule.” Conversely, Jack went from using the
“majority rule” (T1) to dismissing those texts that he could not understand and that clashed with
his prior interpretation. Finally, Mark seemed to move from a provisional, open-ended approach,
sensitive to the cultural context that characterized his performance at T1 to a cautious, but overall
more prior-knowledge dependent reading at T2.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
38
The Classroom Ecology
In order to address our second research question regarding salient aspects of the
classroom environment, we organize the results in four main sections. In the first, we report
results from a structured interview with the teacher, Danielle, regarding her epistemic beliefs in
history (responses to the BHQ), supplemented by her responses to some Grand Tour questions
such as “What is history for you?” that opened the teacher’s interview. In the second, we
summarize Danielle’s performance with regard to historical thinking and reading behaviors on
the second CRT task. In the third, we describe the teacher’s goals, and rationalizations of her
pedagogical practices in relation to the use of primary sources, as they emerged from interviews
and written responses to the teachers’ questionnaire. In the fourth section, we summarize
Danielle’s relevant pedagogical practices, with particular attention to her instruction in relation
to student tasks and activities and the use of primary and secondary sources. In so doing, we rely
on the first author’s fieldnotes of classroom observations and on material distributed in class
(notes, worksheets, tests, and textbooks).
Teacher’s Epistemic Beliefs
“For me [history] is the investigation of the past.” This is how Danielle began
responding to the first author’s Grand Tour question about what history was for her, and it
appeared that much of her thinking revolved around this idea. What drove this investigation?
According to Danielle, both the investigator and the sources at her disposal played a role.
Indicative of this belief was her reaction to the statement “The facts speak for themselves.” She
began by saying that she “almost agree[d],” but then quickly asked “[W]hat constitutes the
fact?”, and answered her question by saying:
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
39
Probably the date an event happened, that’s pretty much a fact. Maybe who was
involved in it, that’s pretty much a fact. So those in and of itself, but it depends on what
you are going to call a fact, when you are approaching a historical event, what are you
calling a fact? Students may take a look and say: “That’s a fact!” Not really, not really.
Danielle was also well aware of several heuristics that facilitate historical thinking. For
example, she realized very clearly that historical accounts have an author and that the warrants of
historical claims are in the evidentiary tracks provided by the historian. In addition, she was not
at a loss in the presence of biased sources, because the reliability of claims could be ascertained
by considering, for example, the author’s purpose in writing the account and by corroborating it
with other sources. Revisiting the idea of what constitutes a historical fact, she mentioned an
activity that she often carried out with the students.
Usually, we use scenarios of something they may be involved in, what is fact and what do
you think it’s opinion? And there are evidences that we are giving them: an object, a
diary. Which would you say is a fact and what an opinion in this particular case? We
have eyewitnesses that say this, but then she is writing something different into the diary,
so which would you rather go for? Which would you believe most?
Although Danielle mentioned the difference between facts and opinions, the entire
interview appears to support the hypothesis that Danielle viewed historical facts as emerging in
the context of a relation between an investigator (who brings questions and perspective to the
table) and some remnants of the past. She was also aware that investigators select among
available evidence and, although she did not give in to the belief that history is just a matter of
opinion, Danielle believed that historical accounts needed to be read critically.
I don’t think that historians go out there [and make up the past], maybe some
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
40
historians have embellished it up a little, but again, I would, I guess, take a look at the
historian’s background, maybe what they have written in the past, their approach, and
again, what sources believed in order to come to that, and be a little skeptical at first.
Danielle would not stop here, though. Since she did not expect “that the historians use all
the available sources,” before using their work she would do some research herself. It is at this
point that an epistemic shift seems to occur. The aspects of Danielle’s thinking reviewed so far
seem compatible with the Criterialist stance. However, other statements from the interview
suggest a more complex view. Specifically, in taking up the role of historian, much more weight
got placed on the objective aspect of knowledge and interpretation came to depend on the
sources at one’s disposal.
It depends on what documents you are looking at. What type of evidence you have will
lead your interpretation of that particular history. You will need to look at all aspects of
it by having, I guess, you will need more reliable sources, depending on the
interpretation of the events.
Thus, good inquiry came to be defined by the sources that one gathers, under the
assumption that a disciplined method of inquiry would give more weight to accounts coming
from “those that actually experienced it.” This attitude toward “digging deeper” seems to imply
an ultimate desire to find out about the past in some uncontaminated and certain form; at the
same time, the awareness that the remnants of the past that we may find are intrinsically biased
left the question about the relation between objective and subjective aspects of historical
knowledge still open, as this quote illustrates:
For me history is never just the facts, because you don’t really know what occurred
unless you go in and research it, and then you know for sure what happened during the
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
41
time period. So you can gather information about a particular historical event, up to a
certain point, but depending on the documents that you pick or the people that you talk
to, there will always be bias. I guess….
Teacher’s Performance
Danielle’s performance on the CRT was overall consistent with several aspects of her
thinking that emerged from the analysis of the interview. In particular, Danielle often
demonstrated awareness of the author of the texts. This emerged clearly after reading Document
3 and Document 4. Danielle read the whole reference and, not knowing much about the authors
of these documents, she focused on the titles of the works and found in them confirmation of her
hypothesis that the documents addressed the conflict of religion versus science at a particular
point in time: “Science and theology, just the title where this is coming from, ‘warfare of science
with theology’. You think religion versus science, coming out again.”
Danielle started out by doing some rereading of the documents, going back for the first
and the second documents to look again and verify her impressions. For the remaining
documents, she moved more directly to an interpretation, summarizing what she saw the
document as contributing to answering the question. After reading the first document, she
identified the theme of science versus religion, which she noted again after reading the third and
fourth documents. For documents 2, 3, 4, and 5, she focused on what the document could tell
her about beliefs relating to the shape of the earth, and basically accepted the documents as
accurate accounts. However, for the sixth document (which calls into question the viewpoint of
a conflict between religion and science with regard to the shape of the earth), her comment was,
“So, they flattened the medieval globe. It is just another person’s viewpoint, as far as –
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
42
document 6.” Thus, although she noted that some conflict between science and religion was
emerging from multiple documents, her intertextual reading stopped here.
Once she finished reading all six texts, she suggested that, in order to answer the
question, she would “probably make a chart, either/or, sphere or flat, and see how many
documents support[ed] the answer; […] or even make a t-chart, science/religion.” In the end, she
“would take more the scientific viewpoint than the theological viewpoint,” because she did not
“really have a lot of evidence to support that the world is flat, the theological standpoint.”
Overall, in building her answer, Danielle seemed to follow a sort of “majority rule,” espousing
the view of the majority of the sources.
Danielle was not particularly satisfied with the kind of knowledge she was able to build
on the basis of the documents and felt that she “would need to look into the theological problem,
why they are going against what science is providing.” In particular, she believed that such lack
of prior knowledge impeded her from evaluating whether some of the documents’ assertions
were a consequence of bias. Hence, constrained by the task to rely only on the texts provided,
Danielle’s process of knowledge construction was brought to a halt by lack of corroboration
across sources, which made it impossible to determine the trustworthiness of specific documents
whenever the author was not previously known.
Teacher’s Goals
History had a profound personal significance for Danielle. She loved investigating her
family history and knowing where she came from. On the other hand, she also believed that
“history cycles” and “it tends to repeat itself.” For this reason, one of her goals for her students
was to understand connections across time and with their personal experience. In the former
respect, this meant being able to identify what changed and what remained the same: “America
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
43
has changed; there have been periods in which history has repeated itself, and has repeated in a
particular way. So, what changed? That particular part is what I try to connect.” In respect to
connections with personal experience, Danielle believed that this was a key factor in making
students interested in what was being studied, in “turning them on.” She referred to a very
positive experience with a low level class in a technology program that got very interested in the
World Fair; the occurrence of 9/11 also sparked interest in the history of the United States.
On the Teacher Questionnaire, Danielle stated that her major goals in teaching history
regarded introducing primary documents with the aim of generating interest and enabling
students to experience “living” history. An associated goal regarded the creation of engaging
lessons fostering more active learning. Introducing primary sources in the curriculum was
conceived as an attempt at sparking some situational interest with the hope of increasing
students’ engagement with the subject matter, which was generally low. In addition, Danielle
used primary sources as facilitators of learning: “So, in reading the documents then, instead of
coming from me or the textbook, it’s coming from an actual person who was there. Maybe it
will stick with them a little better.” While Danielle had often experienced that bits of information
tended to “stick” better when conveyed through primary sources, she was also aware of the risks
of this approach. In fact, students tended to remember “not the actual documents, but quotes,”
and sometimes they reinforced prior misconceptions.
In addition, Danielle believed that making students build understanding from primary
documents would make them “think it through.” While Danielle had found this approach
successful in the past, she was struggling with her current classes, since a high percentage of
students were not willing to work on the assignments and expected her to provide “the answers”.
She also reported that, in this situation, group work was also not providing any incentive to
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
44
thinking, because “it [was] not a collective brain power on it, focused on the information; it
[was] more ‘I sit around and maybe they shall give me the answers’”.
Pedagogical Practices
Use of primary and secondary sources. Although Danielle used a variety of activities
and resources, including maps, documentaries and movies, games, and supplementary reading,
most of the tasks that students completed in Danielle’s class involved some kind of primary
sources. Danielle introduced several primary and secondary sources during her lectures; she
often showed pictures and various representations of census data (or other data pertinent to the
topic). She also consistently referred to some piece of evidence to support her statements and
prompted students to do the same, often asking them to quote evidence from the texts under
discussion. For example, in analyzing a political cartoon by Joseph Keppler, titled “The Bosses
of the Senate,” Danielle asked students who the large figures in the top hats represented. A
student answered that they were the big corporations, who, with all their money, were the owner
of the Senate. Danielle’s question followed: “How do you know?” Students volunteered that
“They are in money bags,” and that “The others are very small.” Danielle kept inviting students
to look at the body language and to observe closely the particulars of the drawing. Yet, Danielle
very rarely shared with students why she selected certain sources and how she found them, why
she thought they were reliable, or, more generally, what process was she following in building a
narrative based on them.
Although different perspectives were often considered and contrasted (e.g., Rockefeller’s
views of industry and popular perceptions of big corporations; Wilson’s point of view and
Zimmerman’s perspective), the process of analysis of sources was seldom explicitly addressed.
From prior years, students should have been familiar with the APPARTS strategy, an approach
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
45
that guided their analysis of a document (textual or pictorial) by noting the author (A), place and
time (P), their prior knowledge of the topic (P), audience (A), reason (R, i.e., why the document
was written), the main idea (T), and the significance of the document (S). Only in one case, the
first author observed that students were explicitly directed to use it. Half of the class seemed to
remember the strategy with no difficulty; however, its application to the specific case (a political
cartoon) was very difficult, since there was no clear indication of where the cartoon was
originally published, and of who its author was.
In one other case, the first author observed a brief discussion regarding the reliability of
sources. Specifically, students had contrasted two accounts of the Pullman strike, identified
whether the authors were for or against the Pullman Company, compared their reports about the
company’s levels of profit and control and their descriptions of the workers’ living conditions,
and considered the sources used to write the accounts. In reviewing the assignment, Danielle
prompted students to quote directly from the accounts in justifying their answers and students
tended to do so with little difficulty. They also easily identified the sources used by the
historians in building their accounts. Then Danielle asked students what account they found
most convincing. When students unanimously decided that it was account A, the one presenting
the most varied sources (as Danielle had previously noted), Danielle asked:
Danielle: How would you label the sources of historian B?
Student: Biased.
Danielle: Yes, they are all from Pullman’s.
This short exchange concluded the evaluation of the two accounts.
In a few cases, students worked in groups (usually using a “jigsaw” approach) to answer
a key question on the basis of multiple sources. For example, students examined the
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
46
government’s role in the settling of the West by reading five sources: President Andrew
Jackson’s Message to Congress “On Indian Removal,” Pacific Railway Act, Homestead Act,
Treaty of Fort Laramie, and Dawes Act. Only the first two documents included excerpts from
the actual source. All the remaining documents had only a description of the content of the
source. Danielle asked students to identify one key word that could capture the government’s
role suggested by each document (e.g., relocation, assimilation); the focus remained on the
content of the texts, and there was no discussion about the different nature of these sources. In
addition, each student read only one of the documents and, although students were supposed to
teach each other about the source they read, most students ended up by copying from each other
the “key words.”
Danielle also introduced multiple primary and secondary sources for integrating with the
textbook and deepening the understanding of specific topics. For example, students read
excerpts from “The Jungle” and from the Meat Inspection Act, together with a brief paragraph on
the supply curve while studying the Progressive Reforms. They also answered a few questions
revisiting some key ideas explained in the readings. In this particular case, the first author was
surprised by the speed (about 10 minutes) with which students read these texts (roughly 4 pages
in length) and answered the questions; Danielle interpreted it as a normal occurrence with this
group and read it as a positive indicator of students’ reading abilities.
Second Research Question: Summary of Findings
What aspects of the classroom environment may have played a role in fostering the
changes in students’ reading of multiple history texts, their ability to think historically, and their
domain-specific epistemic beliefs? First of all, we found that Danielle’s goals and pedagogical
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
47
practice were fairly aligned with her epistemic beliefs, her ability to think historically, and her
overall approach to texts.
Danielle clearly believed that the knower has to play a constructive role in the generation
of historical knowledge. When placing herself in the shoes of the historical investigator,
Danielle also conveyed the belief that such role may be a positive one, since she felt that she was
able to construct reliable knowledge about the past (given enough time to research a specific
issue). Her lectures grounded in evidence may be interpreted as successful examples of how
historical knowledge can be constructed. Yet, similarly to her students, Danielle continued to
view reliability as an intrinsic feature of the sources and not as an evaluation made on the basis
of the question asked by the historian. As such, historical knowledge remains only a matter of
getting to the “right sources”; whenever this is not possible, or once a student comes to the
realization that even the “right sources” are biased, history becomes impossible. This kind of
reasoning emerged very often during students’ structured interviews.
We find that the experiences that these students had in the classroom are actually in line
with this kind of beliefs. For example, as a consequence of placing reliability in the source, the
critical evaluation of texts tended to be reduced to identifying bias, with the implicit goal of
discarding biased sources and relying (or searching for further) unbiased sources. Once a source
could be perceived as overall unbiased, Danielle seemed to take the texts at face value and
consider them as conveyors of information.
Class observations provided evidence of how this belief was translated into pedagogical
practice. We found the (brief) discussion of the two texts about the Pullman strike illuminating,
in this respect. Similarly, when Danielle asked students to address a specific historical question
on the basis of multiple texts (e.g., government role in setting the West), the focus remained on
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
48
the “key words” that students could extract from each text. No attention was paid to the features
of these very different texts; they were equally used as providers of information on a specific
issue. Students’ think-alouds offered ample evidence of this hampering approach to text.
Within this framework, even a powerful heuristic such as sourcing was reduced to the
identification of possible biases and turned out to be not particularly helpful. In fact, in the CRT
task, Danielle remained unable to evaluate those texts whose authors were not familiar to her. In
line with this finding, when students were prompted to look at the authors of a source, it was
mainly to determine their biases. Students’ increased reference to issues of bias is thus in line
with what observed in the classroom. In Mark’s case, his perceptions of certain sources as
biased gravely impeded understanding. However, discussions of texts’ authors were rare
occurrences and, in most cases, primary and secondary sources were offered as an integration of
the textbook narrative. Not surprisingly, with the exception of Mark, treating the documents as
authorless texts was a common feature of students’ approach to the CRTs.
Students’ increased reliance on the “majority rule” to decide in situations of conflict
among multiple sources also found a parallel to Danielle’s suggestion of dealing with the CRT
by making a T-chart. Putting together a narrative by piecing together various pieces of texts was
also in line with most of the tasks that asked students to work with multiple sources.
Final Remarks
Although we do believe that this study offers an original contribution to extant literature,
the interpretation and generalization of these results is constrained by several limitations. First,
we had to be respectful of the goals and demands of the particular learning context of the
classroom. For example, the choice to limit to four the number of informants in each class was
dictated by the inopportunity to take students out of the class for interviewing if the teachers felt
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
49
that missing a specific lesson could significantly impact their learning experience and success in
that class, a concern that was especially high at the beginning of the semester. Thus, the results
reported in this study are based on data collected from a small, partially self-selected sample of
student informants. In addition, our conceptualization of these constructs surely influenced the
analysis of the data, drawing some students’ and teacher’s ideas to the forefront and leaving
others in the background. Although we tried to avoid or at least control an unjustified
subjectivity, the results of the study are clearly limited by our own understandings.
Although our study focused only on the classroom’s context, it would be nearsighted to
dismiss the influence that the school system as a whole may have played in several of Danielle’s
choices and, more broadly, in creating a culture that fostered and made some of her beliefs
particularly adaptive. It is also far from our intention to imply that Danielle was less than a
caring and hard-working professional. Despite the fact that she was teaching 180 students, her
lessons were carefully planned, homework was daily checked and returned to students, and she
used a broad repertoire of tasks and assignments. Yet, by focusing on the classroom context, we
believe that this study does contribute to shedding light on the formation of domain-specific
epistemic beliefs, and the development of specific approaches to texts in general and to the
historical text in particular.
The differences we found across students in terms of history-specific epistemic beliefs
suggest that the classroom is not the only context influencing the formation of students’ beliefs.
Yet, the trends in change that we were able to detect across a relatively short period of time
indicate that classrooms play a key role in this respect.
The influence that classrooms can have on the development of reading and of historical
thinking appears to be even greater, especially considering that students differed considerably
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
50
across a number of additional cognitive and motivational variables. The similarities in
performance on the CRT across students and between students and their teacher suggest that
historical thinking and critical reading of texts do not develop by themselves, but require a
purposeful and committed education.
Perhaps the main but unsettling contribution of this study lies in the finding that students
did learn criteria to think in history and consolidated or acquired reading habits in approaching
multiple texts; unfortunately, these were often hindering their development and hampering their
encounter with texts. In a context characterized by abundant and often conflicting information,
the lack of criteria and “efficient,” but harmful reading habits are especially worrisome. Mark,
who was an exceptional and in many respects ideal student, stated that he would rely on the
internet to continue his inquiry; yet, he was at a loss when asked about how he would make sense
out of his potential findings. It is hard to justify that it has to be this way.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
51
References
Afflerbach, P., & Cho, B. (2009). Responsive comprehension strategies in new and traditional
forms of reading. In S. E. Israel & G. G. Duffy (Eds.), Handbook of research on reading
comprehension (pp. 69-90). New York: Routledge.
Bain, R. (2000). Into the breach: Using research and theory to shape history instruction. In P. N.
Stearns, P., Seixas, & S. Wineburg (Eds.), Knowing, teaching and learning history:
National and international perspectives (pp. 331-352). New York: New York University
Press.
Bain, R. (2005). “They thought the world was flat? Applying the principles of how people learn
in teaching high school history. In S. Donovan & J. Bransford (Eds.), How students
learn: History in the classroom (pp. 179-213). Washington D.C.: The National
Academies Press.
Bråten, I. (2008). Personal epistemology, understanding of multiple texts, and learning within
internet technologies. In M. S. Khine (Ed.), Knowing, knowledge, and beliefs:
Epistemological studies across diverse cultures (pp. 351-376). United States: Springer.
Bråten, I., & Strømsø, H. I. (2003). A longitudinal think-aloud study of spontaneous strategic
processing during the reading of multiple expository texts. Reading and Writing: An
Interdisciplinary Journal, 16, 195-218.
Brickhouse, N. W., Dagher, Z. R., Shipman, H. L., & Letts, W. J. (2002). Evidence and warrants
for belief in a college astronomy course. Science & Education, 11, 573-588.
Cacioppo, J., Petty, R., Feinstein, J., & Jarvis, B. (1996). Dispositional differences in cognitive
motivation: The life and times of individuals varying in need for cognition. Psychological
Bulletin, 119(2), 197-253.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
52
Elby, A. (2001). Helping physics students learn how to learn. Phys. Educ. Res., Am. J. Phys.
Suppl., 69(7), 54-64.
Fox, E. (2009). The role of reader characteristics in processing and learning from informational
text. Review of Educational Research, 79, 197-261.
Gottfried, A.E. (1985). Academic intrinsic motivation in elementary and junior high school
students. Journal of Educational Psychology, 77(6).
Harackiewicz, J., Barron, K., Tauer, J., Carter, S., & Elliott, A. (2000). Short-term and long-term
consequences of achievement goals: Predicting interest and performance over time.
Journal of Educational Psychology, 29(2), 316-330.
Hartman, D. K. (1995). Eight readers reading: The intertextual links of proficient readers reading
multiple passages. Reading Research Quarterly, 30, 520-561.
Hofer, B. K. (2004). Epistemological understanding as a metacognitive process: Thinking aloud
during online searching. Educational Psychologist, 39(1), 43-55.
Husbands, C., Kitson, A., & Pendry, A. (2003). Understanding history teaching: Teaching and
learning about the past in secondary schools. Philadelphia: Open University Press.
Hynd, C., Holschuh, J. P., & Hubbard, B. P. (2004). Thinking like a historian: College students’
reading of multiple historical documents. Journal of Literacy Research, 36(2), 141-176.
Kendeou, P., & van den Broek, P. (2005). The effects of readers’ misconceptions on
comprehension of scientific text. Journal of Educational Psychology, 97, 235-245.
King, P. M., & Kitchener, K. S. (2002). The Reflective Judgment Model: Twenty years of
research on epistemic cognition. In B. K. Hofer, & P. R. Pintrich, (Eds.), Personal
epistemology: The psychology of beliefs about knowledge and knowing (pp. 37-61).
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
53
Kuhn, D., & Weinstock, M. (2002). What is epistemological thinking and why does it matter? In
B. K. Hofer, & P. R. Pintrich, (Eds.), Personal epistemology: The psychology of beliefs
about knowledge and knowing (pp. 121-144). Mahwah, NJ: Lawrence Erlbaum
Associates.
Lee, P., & Ashby, R. (2000). Progression in historical understanding among students ages 7-14.
In P. N. Stearns, P. Seixas, & S. Wineburg (Eds.), Knowing, teaching and learning
history: National and international perspectives (pp. 199-222). New York: New York
University Press.
Lee, P., Dickinson, A., & Ashby, R. (1997). “Just another emperor”: Understanding action in the
past. International Journal of Educational Research, 27, 233-244.
Lee, P., & Shemilt, D. (2003). A scaffold, not a cage: Progression and progression models in
history. Teaching History, 113, 13-24.
Maggioni, L., Alexander, P. A., & VanSledright, B. (2004). At a crossroads? The development
of epistemological beliefs and historical thinking. European Journal of School
Psychology, 2(1-2), 169-197.
Maggioni, L., Fox, E., Alexander, P. A. (2009, April). Adolescents’ reading of multiple history
texts: An interdisciplinary exploration of the relation between domain-specific reading
and epistemic beliefs. Paper presented at the Annual Meeting of the American
Educational Research Association, San Diego.
Maggioni, L., VanSledright, B., & Alexander, P. A. (2009). Walking on the borders: A measure
of epistemic cognition in history. The Journal of Experimental Education, 77(3), 187214.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
54
Midgley, C., Kaplan, A., Middleton, M., Maehr, M., Urdan, T., Anderman, L., Anderman, E., &
Roeser, R. (1998). The development and validation of scales assessing students’
achievement goal orientations. Contemporary Educational Psychology, 23, 113-131.
Muis, K.. R. (2007). The role of epistemic beliefs in self-regulated learning. Educational
Psychologist, 42(3), 173-190.
Paxton, R. J. (2002). The influence of author visibility on high school students solving a
historical problem. Cognition and Instruction, 20(2), 197-248.
Perfetti, C. A., Rouet, J., & Britt, M. A. (1999). Toward a theory of documents representation. In
H. van Oostendorp & S. R. Goldman (Eds.), The construction of mental representations
during reading (pp. 99-122). Mahwah, NJ: Lawrence Erlbaum Associates.
Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively
responsive reading. Hillsdale, NJ: Lawrence Erlbaum Associates.
Pritchard, R. (1990). The effects of cultural schemata on reading processing strategies. Reading
Research Quarterly, 25, 273-295.
Rouet, J. F., Britt, M. A., Mason, R. A., & Perfetti, C. A. (1996). Using multiple sources of
evidence to reason about history. Journal of Educational Psychology, 88(3), 478-493.
Rouet, J., Marron, M. A, Perfetti. C., A., & Favart, M. (1998). Understanding historical
controversies: Students’ evaluation and use of documentary evidence. In J. Voss & M.
Carretero, M. (Eds.), International review of history education, Volume 2: Learning and
reasoning in history (pp. 95-116). London: Woburn Press.
Ryan, M. P. (1984). Monitoring text comprehension: Individual differences in epistemological
standards. Journal of Educational Psychology, 76(2), 248-258.
Ryder, J., Leach, J., & Driver, R. (1999). Undergraduate science students’ images of science.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
55
Journal of Research in Science Teaching, 36(2), 201-219.
Shanahan, C. (2009). Disciplinary comprehension. In S. E. Israel & G. G. Duffy (Eds.),
Handbook of research on reading comprehension (pp. 240-260). New York: Routledge.
Strømsø, H. I., & Bråten, I. (2002). Norwegian law students’ use of multiple sources while
reading expository texts. Reading Research Quarterly, 37, 208-227.
Strømsø, H. I., Bråten, I., & Samuelstuen, M. S. (2003). Students’ strategic use of multiple
sources during expository text reading: A longitudinal think-aloud study. Cognition and
Instruction, 21, 113-147.
VanSledright, B. (2002a). In search of America’s past: Learning to read history in elementary
school. New York: Teachers College Press.
VanSledright, B. A. (2002b). Fifth graders investigating history in the classroom: Results from a
researcher-practitioner design experiment. The Elementary School Journal, 103, 131-160.
VanSledright, B. A. (2004). What does it mean to read history? Fertile ground for crossdisciplinary collaborations? Reading Research Quarterly, 39(3), 342-346.
Wigfield, A., Eccles, J., Yoon, K., Harold, R., Arbreton, A., Freedman-Doan, C., Blumenfeld, P.
(1997). Change in children’s competence beliefs and subjective tast values across the
elementary school years: A 3-year study. Journal of Educational Psychology, 89(3), 451469.
Wineburg, S. (2001a). Historical thinking and other unnatural acts: Charting the future of
teaching the past. Philadelphia: Temple University Press.
Wineburg, S., & Wilson, S. (2001). Wrinkles in time and place: Using performance assessment
to understand the knowledge of history teachers. In S. Wineburg, (Ed.), Historical
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
56
thinking and other unnatural acts: Charting the future of teaching the past (pp. 173-214).
Philadelphia: Temple University Press.
Wolfe, M. B. W., & Goldman, S. R. (2005). Relations between adolescents’ text processing and
reasoning. Cognition and Instruction, 23, 467-502.
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
57
Appendix A
Historical Thinking Rubric
CODE
DESCRIPTION
EXAMPLE
HTyes
Evidence of use or knowledge
of heuristics signaling
historical thinking.
“You have to know where a source is coming from to
understand that it’s biased.”
Historical
Thinking
Yes
HTno
Historical
Thinking
No
CP
Cut and
Paste
AQ
Awareness
of the
Question
AA
Awareness
of the
Author
“Usually, I compare them and see how they differ.”
Evidence of use or knowledge
of heuristics clearly
incompatible with historical
thinking.
“I don’t even know what the historical method is and I can know
history well, kind of.”
Selecting more or less
arbitrarily parts from different
documents in order to build a
more or less coherent story (no
intertextual comparison;
dismissal of conflicting
evidence)
“They believed that they, he was Lono, their great god that had
promised to return and finally returned on his floating island and
they believed it so much that they worshiped him as an actual
god and not as a men, because as he said in document they
wouldn’t have done it for another human being, but what they
gave him, gold, and sacrifice, and lot of stuff, lot of a great
stuff.”
Building an answer to the task
question
“So these people believed in a god who had already been there
and that should come back.”
“I didn’t read it [the references]. I don’t know, it just it doesn’t
seem, it’s not just like the same distance and font, and it’s all
together and that’s separate, so like kind of, it is where it came
from, I guess, it has the cite, so to me, where it came from, it
could be a rumor or not, so to me it does not affect how I wrote
my paper or anything like that, so I just never read that.”
“So this priest pretty much knew that they were not gods, but to
be sure that they were not gods he pretended that they were
gods.”
Awareness of author (in the
text). Usually signaled by use
of personal pronouns (e.g., he;
they)
“So, he’s making an opinion, he thinks that they made it up.”
“How do they know it was in sacrifice and not just a gift?”
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
58
Appendix B
Epistemic Beliefs Rubric
CODE
DESCRIPTION
EXAMPLE
EBCO
Evidence is seen as detached from
argument.
In other words, there is no overall
awareness of the role of the knower.
“There shouldn’t be some method of inquiry for history it
should just be what it is and method could skew the
result.”
Copier
TR1
Transition 1
EBSUB
Subjectivist
TR2
Transition 2
EBCR
Criterialist
Ideally, history should coincide with
the past. However, since we cannot
know all of it, whenever the evidence
is debatable or simply cannot be
found, it remains a matter of
opinions.
(historian as “wanna be” or “should
be” chronicler)
Another manifestation is the
dichotomy facts vs. opinion. Facts
are objective, while opinions cannot
be challenged.
Clear predominance of the subject;
history is unjustified and biased.
Focus is mainly on the knower
History depends on one’s opinions
that color how one judges history and
how one makes selections (e.g.,
political opinions)
Historian’s opinion are unbounded
by evidence, because there is no
evidence or it does not really matter.
History is the interpretive work of
the historian based on evidence; the
existence of a method is
acknowledged, but there is no clarity
about how it may look like. In other
words, the dynamic subject/object is
acknowledged but there is no specific
reference to a method.
History is the interpretive work of
the historian based on evidence;
interpretation relies on specific
disciplinary criteria. Students are
aware of what these criteria are
although they may not know how to
use them.
“Historians are just humans, they do not make history,
other people make it, he can just go and tell you how it
goes.”
“You really don’t know history, it’s just through books
and people writing down stuff and documents from back
in the days; there could be something missing that
nobody knows about, but as people go, everybody has a
different opinion about history and what they think
happened.”
“Everyone should have their own opinion, as long as you
know the solid facts.”
“History is basically what you make of it depending on
what you have got to know, what your background is,
like democratic, republican, because history, especially
like that, people see it differently depending on whether
you are republican or democratic.”
“The past is what the historian makes it to be, because
every historian has a different view on how it’s
happened.”
“There is some evidence on something, so they can’t just
choose, they have to actually research the evidence, what
other theories there are out there, so, and there are ways
of knowing, it just takes a while.”
“When you read something, like an historical document
that was written by some of the historian, you need to
understand and read between the lines to understand what
he is saying and to understand what he or she is trying to
do”
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
59
Appendix C
Reading Behavior Codes
Single text codes (drawn from Pressley & Afflerbach, 1995)
CODES
EXAMPLES
Strategic behaviors
Re-reading
Guessing the meaning of a word in context
Questioning
a. General
b. Word meaning
c. Author
“is it like his name?”
a. “Is that Mark Twain?”
b. “What is a deity?”
c. “My first thought is how did they know what their
chiefest god is?”
Arguing with text
“Why would he be involved in such spiritual ceremonies
if he was just a filthy old man?”
Visualizing
“OK, so I was just doing an image, thinking about an
image.”
Restating or repeating text information
a. local (word, phrase, sentence level)
b. global (paragraph, passage level)
Making connections
a. to background knowledge
b. to personal experience
c. to prior text
d. to research task
a. “They are celebrating their new year festival.”
b. “I think it just discusses the people in their beliefs to
their god.”
a. “Hawaiian, haven’t they that shorter alphabet?”
b. “I used to think that the world was flat.”
c. “So, now the ship is the temple.”
d. “So, they think he was a god? Did they?”
Interpreting (a statement requiring reasoning
beyond information in the text to build text
meaning)
“They believed that he was a god because they are
treating him so good.”
Elaborating (a statement requiring the use of
additional information not explicitly in the
text to build beyond text meaning)
“He was just a great person”
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
Monitoring/Evaluative/Expressive behaviors
Evaluating comprehension (positive or
negative)
“…I don’t understand that.” (negative)
Evaluating text quality (positive or negative)
“this is obviously biased by someone who favored
James Cook”
“Oh, there are three more?”
Evaluating task difficulty
Evaluating agreement with text (positive or
negative)
“I obviously agree with this statement, this one.”
Evaluating interest (positive or negative)
“You see, that is interesting to me right now.”
Expressing amusement
“Which is kind of funny. (laughing)”
60
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
61
Intertextual codes (adapted from Afflerbach & Cho, 2009)
CODES
EXAMPLES
Identifying and learning important information
A1. Relating the current text to recently read
(prior) texts.
“So, the first document is saying that it is not round, the
second document that it is round.”
A2. Predicting contents of current text based
on understanding of previously understood
text.
“Document 5, it’s about the Greeks, and I am guessing
what they thought about that.”
A3. Comparing and contrasting the content of
the text being read with the content of related
texts to develop a coherent account of crosstextual contents.
“yeah, the first document.”
A4. Elaborating with information to
understand text contents by connecting ideas
between texts.
“because they saw – where is that – document 3 I think,
they saw the big ship and so probably they did not
usually see, so they made an exception to go out and see
it”
A5. Identifying a theme across multiple texts.
“they are based on pretty much the same thing.”
A6. Attending to an identified theme across
two or more texts to organize this information.
“it is pretty much it, they think he is a god.”
A7. Noting ambiguity in tentative meaning of
texts.
“from the documents it doesn’t really show that there
was a general consensus about the shape of the earth at
the time of C. Columbus.”
“ I would probably go back and reread these, most
likely.”
A8. Building increased understanding of topic
by rereading the information contained in two
or more texts.
A9. Focusing on gist information across
multiple texts to recursively construct meaning.
“So, Still thinking he is a god.”
A10. Linking text segments to finalize crosstextual meaning structures.
“He was just trying to settle a new land, discover new
places.”
A11. Identifying the unique and shared
contributions of information to the constructed
meaning of multiple texts.
“ he also got things from them, like in document, what
was it, five, barter,”
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
62
Monitoring
B1. Regulating meaning construction strategies
according to task and goal.
“after reading the sixth document, yes, it is the last
one, I kind of brought the idea full circle in my mind
and constructed an opinion.”
B2. Perceiving that multiple texts related to the
same topic can provide diverse views about the
topic, complementary information about the topic,
or both.
“there were lots of differences before in thinking in
here that I can write down.”
Evaluating
C1. Using information about the source of a text to
evaluate and interpret text contents.
“ so the source was the Apotheosis, so he’s making
an opinion.”
C2. Perceiving and distinguishing the
characteristics of different texts and evaluating
texts’ accuracy or trustworthiness based on these
features.
“Because it was in first person, so, it kept saying I a
lot and it helps to know, when someone says I a lot
it helps to know who is talking because if there is a
point of view,.”
C3. Conducting a text to text evaluation using a
gestalt impression of each text.
“I obviously agree with this statement, this one,
probably more so than the other ones.”
C4. Evaluate one text in relation to another, using
specific information
“This kind of makes sense with what it said in a lot
of the other articles.”
C5. Evaluate contribution of single text to reading
and task goals.
“The fourth document doesn’t really say too much
about what the Hawaiians thought”
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
63
Appendix D
Individual Student Reading Behavior Codes
Ashley – First CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
CODE
Questioning (word)
2
A5
1
Elaborating
1
Ashley – Second CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
Interpreting
2
C5
Elaborating
1
Connecting - prior knowledge
2
Connecting - task
1
Evaluating interest
1
Questioning (word)
1
Questioning (general)
1
FREQUENCY
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
64
Elizabeth – First CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
FREQUENCY
Restating (local)
3
C5
5
Restating (global)
1
A5
3
Interpreting
2
A4
1
Elaborating
1
A6
1
Connecting - task
2
Rereading
2
Elizabeth – Second CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
FREQUENCY
Interpreting
10
C5
6
Elaborating
3
A6
4
Restating (local)
6
B2
3
Restating (global)
2
C3
2
Connecting – prior knowledge
2
A1
1
Connecting – prior text
1
A2
1
Connecting - task
1
A3
1
Questioning (general)
3
A5
1
Evaluating comprehension (-)
1
C2
1
Rereading
1
Expressing amusement
1
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
65
Jack – First CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
FREQUENCY
Restating (local)
14
C5
5
Restating (global)
2
A10
1
Interpreting
5
Elaborating
1
Connecting – prior knowledge
2
Questioning (word)
1
Jack – Second CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
FREQUENCY
Restating (local)
8
C5
2
Restating (global)
3
A6
1
Interpreting
2
A9
1
Elaborating
3
Questioning (word)
2
Guessing word meaning
1
Connecting – prior knowledge
1
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
66
Mark– First CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
FREQUENCY
Questioning (author)
10
B2
3
Questioning (general)
2
B1
1
Questioning (word)
1
C2
1
Arguing
3
C3
1
Elaborating
6
Connecting - task
1
Rereading
1
Evaluating text quality (-)
1
Evaluating agreement (+)
1
Evaluating comprehension (+)
1
Visualizing
1
Mark – Second CRT
Single Text
Intertextual
CODE
FREQUENCY
CODE
FREQUENCY
Interpreting
17
A9
3
Elaborating
8
A3
1
Rereading
8
A4
1
Restating (local)
5
A6
1
Restating (global)
3
A7
1
Evaluating comprehension (-)
3
A8
1
Evaluating comprehension (+)
3
A10
1
Connecting – prior knowledge
2
B2
1
Connecting – experience
1
C1
1
Connecting - task
2
C2
1
Evaluating task difficulty
2
C4
1
Questioning (word)
1
C5
1
Questioning (general)
1
Evaluating agreement (+)
1
Running head: WHEN CHANGE DOES NOT MEAN PROGRESS
Evaluating text quality (-)
1
Expressing amusement
1
67