Undergraduates’ Reading Success
1
Factors Associated with Undergraduates’ Success in Reading and Learning from Course Texts
Emily Fox
Daniel L. Dinsmore
Liliana Maggioni
Patricia A. Alexander
University of Maryland
Paper to be presented at the Annual Meeting of the American Educational Research Association,
April, 2009, San Diego
Undergraduates’ Reading Success
Abstract
This study reports a mixed-methods exploration (N = 61) of factors associated with
undergraduates’ more or less successful learning from typical college texts. Of interest are
person-based factors including topic-related knowledge/interest, reading approach and need for
cognition, text-based factors including familiarity, accessibility, and interestingness, and their
interaction as manifested in observed processing behaviors and consequent learning from text.
We found that quantitative consideration of single factors or groups of factors benefited from
supplementation with case studies. Application of reading profiles had moderately good fit, but
problematic considerations included use of outcome scores, a role for situational interest, and
moderate levels of relevant variables. Overall, undergraduates were less successful in learning
from more unfamiliar and uninteresting text, but appropriate challenge also related to
engagement and learning.
2
Undergraduates’ Reading Success
3
Factors Associated with Undergraduates’ Success in Reading and Learning from Course Texts
College undergraduates are expected to comprehend and learn independently from
college-level texts across relatively unfamiliar academic domains, acquiring new vocabulary and
key concepts and procedures, as well as principles of reasoning (Simpson & Nist, 2002). To be
successful in college, these students must be both willing and able to “read with comprehension
even when the material is neither easy to understand nor intrinsically interesting” (RAND
Research Study Group, 2002, p. xiii).
How successful are undergraduates at this type of reading? Can they read independently
even when it involves effort and confrontation with unfamiliar material? We suggest that many
undergraduates may not have the cognitive and motivational tools to equip them for college-level
reading, but may come to situations involving learning from text burdened by such factors as a
passive or even resistant approach to reading, an over-reliance on interest, background
knowledge or personal experience, or a lack of strategic flexibility (Fox, Dinsmore, &
Alexander, 2007; Fox, Dinsmore, Maggioni, & Alexander, 2008; Roberts & Roberts, 2008).
In this investigation we explore factors that may be associated with undergraduates’
success in learning independently from text by looking at how these students approach reading
new material in their course textbook that may be challenging in its perceived accessibility or
interest as compared to their reading of possibly more accessible or interesting textbook material
that has already been covered in class. Of interest here are person-based factors including topicrelated knowledge and interest, reading approach and need for cognition, along with text-based
factors including familiarity, accessibility, and interestingness, and the interaction of these
factors as manifested both in strategic and evaluative/monitoring behaviors observed via thinkaloud protocols and in consequent learning from text. Although undergraduates are familiar
Undergraduates’ Reading Success
4
participants in studies of reading, there is need for both greater detail and more breadth in
consideration of the factors involved in how these students read and the interaction of these
factors in a given reading situation (Simpson & Nist, 2002). Our mixed-methods approach is
intended to capitalize on the explanatory affordances of both quantitative comparisons and
qualitative richness of description and categorizations; we believe consideration of both types of
patterns will be informative in attempting to construct a more or less unified explanation of this
complex phenomenon.
Theoretical Framework
The constructive activity of reading comprehension has been conceptualized as involving
the interrelated elements of reader, text, and activity or purpose for reading (RRSG, 2002). Our
consideration of undergraduates’ relative success in reading is built around these three elements
as its framework.
Activity
College students are expected to be able to read extensively and independently in
domains to which they may still be becoming acclimated and in which they may never become
experts. According to Simpson & Nist, “college students are expected to understand and
remember what they read with few supports and less guidance.…we discovered from a survey of
more than 223 professors on our campus that more than 60% expected their students to master
textbook concepts through independent reading because they were not going to discuss the
contents during class” (2002, p. 365). The activity of reading, understanding, and remembering
the gist of assigned textbook material that may vary widely in its relevance to the student’s longrange educational goals or appeal to immediate interest is a typical task and purpose for collegelevel reading. However, situating our investigation of undergraduates’ relative success in reading
Undergraduates’ Reading Success
5
in the context of such an activity has a number of consequences. We will be, in a sense, damping
down the possible effects of interest and knowledge by deliberately not targeting the type of
reading that is undertaken by students as they learn about an area in which they hope to advance
in expertise, as a chosen academic or career goal. Although such reading and students’ success in
this more obviously relevant reading are clearly of great importance, it is also the case that
undergraduates in many courses of study must also take more general-purpose courses and do
more general-purpose reading. In such cases, reader-based and text-based factors other than
topic-related knowledge and interest, such as reading approach, need for cognition, and text
accessibility, familiarity, and interestingness, may emerge with a stronger role in reading
success.
Reader
The more apparent differences between successful and unsuccessful college students, for
example, have been found rather in how the student reads than in what he reads. The
successful student is marked by his disposition to read beyond assignments, to re-read,
and to apply his reading to his experience. The reader’s total background of training and
experience determines what meanings he will assign to particular words and passages;
wide variations in such meanings have been revealed in several studies. And, finally, his
previous knowledge and his previous attitudes influence his whole understanding of and
response to the publication. (Waples, Berelson, & Bradshaw, 1940, p. 83-84)
What reader characteristics are likely to be associated with more or less successful
learning from text in the type of general-purpose reading activity described above? The above
quote from Waples, Berelson, and Bradshaw suggests that identification of specific
characteristics is likely to be a daunting task. However, we do know that successful engagement
Undergraduates’ Reading Success
6
in the constructive activity of reading and learning from text is likely to depend on how much the
reader already knows about the topic under consideration, on how motivated the reader is to
exert any necessary effort, and on the reader’s capabilities in terms of strategic processing
(RRSG, 2002). We know that constructing knowledge from text requires some degree of
background subject-matter knowledge, and proceeds more effectively for readers who possess
such knowledge (Ferstl & Kintsch, 1999; McKeown, Beck, Sinatra, & Loxterman, 1992). We
expect that more principled, integrated, and coherent knowledge is more effective in facilitating
reading and learning from text, and also that more knowledge, whether coherent and integrated
or not, is likely to be more effective than less knowledge (barring misconceptions or conflicts
with text material). However, an interesting question here is the role of fragile knowledge that
may not be strongly integrated. If a reader has only fragmented bits of knowledge, how might
such bits come into play in facilitating more successful learning from text that makes use of
those bits in building a integrated and coherent explanation or description of the relevant
phenomenon? In a generally knowledge-lean situation, does a reader who knows just a little have
a significant advantage over a reader who knows even less?
A given reading situation will also tend to call for more or less effort from a reader, and
readers may be more or less motivated to engage with a particular text due to a number of
possible factors, including: initial level of interest in (or antipathy toward) the text topic or
situationally sparked interest in some segment of the text (Schiefele, 1999) and dispositional
tendency to engage in cognitive effort in general, or need for cognition (Cacioppo, Petty,
Feinstein, & Jarvis, 1996; Dai & Wang, 2005).
Specific views of reading and learning from text as requiring deeper or shallower
processing have been associated with deep and surface approaches to learning (Marton & Säljö,
Undergraduates’ Reading Success
7
1997; Säljö, 1997). The processing of readers with a surface learning approach tends to be
atomistic, local, and passive, while that of readers with a deep learning approach is more holistic,
global, and actively dialogic (Marton & Säljö, 1997). Marton & Säljö (1997) found that these
different approaches are related to how students understand the nature of learning from text and
learning in a college-level course, as well as to the outcomes at which they arrive. Säljö (1997)
points out that along with a deep processing approach tends to go a commitment to engagement
in appropriate effort when reading, even in situations where the reading was not self-chosen or
the desirability of the knowledge to be gained is not immediately apparent.
In such situations, reading is characterized by a voluntary and self-induced decision to
attend to a written discourse in which there is a genuine and momentary desire to find out
what is ‘made known.’ A basic feature of a deep approach therefore seems to be that this
attitude is also maintained in a situation where there may not be such an initial
commitment on the part of the reader, but where the reading is undertaken in response to
a request or requirement. (Säljö, 1997, p. 103)
The success of readers’ efforts at comprehending and learning will vary as well depending on
their declarative, procedural, and conditional knowledge related to reading, which will determine
whether they can flexibly enact appropriate reading strategies when necessary (Paris, Wasik, &
Turner, 1991).
Another way of considering readers’ likely success in learning from text in a given
reading situation based on their consistent approach to reading across situations is given by
Alexander (2005). Alexander identified a set of six reader profiles or consistent approaches to
reading that help us understand patterns of reading success in terms of background knowledge,
reading-related knowledge, knowledge and use of strategies, interest in reading, and likely
Undergraduates’ Reading Success
8
engagement in a given reading situation: Highly Competent readers, who have strong interest in
and knowledge of reading; Effortful Processors, who have high engagement in reading, but must
often exert considerable effort to build understanding; Knowledge-Reliant readers, who can
engage successfully with texts for which they have strong background knowledge or personal
experience;. Non-Strategic Processors, who do not possess an adequate repertoire of strategies;
Challenged readers, who have lack mastery of fundamental reading processes; and Resistant
readers, who may have the necessary background knowledge, reading knowledge, and strategy
knowledge, but are not interested in reading.
Text
What text characteristics are likely to be associated with more or less successful learning
from text for the readers and reading activity under consideration? Possible factors involved in
this situation of reading selections from the course textbook include familiarity, interestingness,
and accessibility. Greater familiarity of the information presented in the text is likely to promote
more successful knowledge construction (Alexander & Jetton, 2000), although it has the
potential to evoke both positive and negative responses in terms of level of engagement. More
familiar information can be a stimulus to connections to prior knowledge but it can also be
boring, and unfamiliar information can present a welcome challenge or can be a barrier (Kintsch,
1980). Perceived interestingness of text can promote greater engagement and thus possibly better
learning from text (Wade, Buxton, & Kelly, 1999). Perceived accessibility of the text in terms of
difficulty, vocabulary, organization, quality of explanations, and appropriate use of text features
might also tend to be associated with more successful learning from text (Armbruster, 1984).
Perceived uninterestingness or inaccessibility could hinder engagement or even provoke
resistance.
Undergraduates’ Reading Success
9
Summary
This study is a mixed-methods exploration of person-based and text-based factors
associated with undergraduates’ more or less successful learning from typical college texts and
the interaction of those factors. We are interested in how the text-level and person-level factors
outlined above help us to understand what is going on in terms of undergraduates’ relative
success in learning from course-related material as presented in two passages from their
textbook. We expect to see multiple levels of explanations emerging as we take into account
these person-level and text-level factors as they interact in this situation.
Specifically, we will consider how the factors and their interactions help us to account for
relative levels of success in reading and learning from text. We will consider who is successful
on which passages in terms of what we know about the texts and about the readers, looking for
patterns that can reveal the role of individual factors as well as more complex patterns arising
from their interplay.
Methods
Participants
Our participants were 61 (32 female) college undergraduates taking an upper-level
elective course on Research Methods at a four-year university in a mid-Atlantic state. Although
required for several majors, this course itself does not align with any specific major at the
university. It tends to be populated by students with a wide range of interests and academic
capabilities, including some who need an upper-level elective prior to graduation and are looking
for a course that will not have a heavy workload. Reading of their course textbook by such
participants was thought to be a good exemplar of the type of college reading outlined in our
section above on Activity.
Undergraduates’ Reading Success 10
Our participants were predominantly upperclassmen with a wide range of identified
majors, including sociology, English, public health, economics, criminal justice, psychology, and
journalism. Their ages ranged from 18 to 30, with a mean of 21.0. Their reported ethnic
background was reasonably diverse, given the makeup of the student population at the
university, with 49% of participants self-reporting as European American, 23 % as African
American, 11% as Hispanic American, 5% as Asian American, and 7% as other; three
participants (5%) did not respond to this question. Their self-reported college GPAs ranged from
1.60 to 4.00, with a mean of 2.98 (one participant did not respond) and their cumulative college
credits completed ranged from 15 to 156, with a mean of 79 (one participant did not respond).
They thus appeared to be a reasonably representative sample of typical undergraduates at our
university for our reading situation. Participants were enrolled in either one section of Research
Methods in the fall 2007 semester (for which the third author was instructor) or one of three
sections in the spring 2008 semester. They were recruited by the first and second author, and
offered course credit for participation. Participants did not differ markedly on pre-test prior
knowledge scores across these four classrooms, so that we felt comfortable in pooling their data
to form the sample for this study.
Measures
Demographics. Participants completed a demographics questionnaire giving background
information on gender, ethnicity, age, cumulative college credits completed, major, and average
college GPA. We also asked them to briefly describe their reading habits in two open-ended
questions asking them to indicate about how often they read books or other materials outside of
their assigned coursework, and what they typically read.
Undergraduates’ Reading Success 11
Interest. An 8-item interest rating questionnaire assessed interest in topics related to
research methods. Responses were on a 1 to 5 scale, where 1 was Not at all interested and 5 was
Very interested. The topics were taken from the chapters in the course textbook (Cozby, 2007)
and included: scientific method, research design, experimental research, qualitative research,
reliability and validity, statistical inference and analysis, understanding research results, and
evaluating research. Reliability was relatively strong, with a Cronbach’s alpha of 0.76. This
measure has a potential minimum of 8 and a maximum of 40. In our sample, the mean was 27.1
(SD = 4.94), with a range of 14 to 38.
Need for Cognition. Participants completed the 18-item short Need for Cognition (NFC)
scale (Cacioppo et al., 1984), using a 5-point response format in which they indicated for each
statement the degree to which it was characteristic of them, with 1 = Extremely uncharacteristic
of you and 5 = Extremely characteristic of you. Half of these items are reverse coded, and higher
scores indicate a higher need for cognition. Reliability for this measure was strong, at α = 0.85.
This measure has a potential minimum of 18 and a maximum of 90. In our sample, the mean was
61.22 (SD = 10.11), with a range of 41 to 82.
Knowledge. Participants completed an 18-item multiple choice assessment of their
overall knowledge of research methods, and two 10-item multiple choice assessments of their
knowledge of the topics of statistical inference and the scientific approach. The items used were
taken from the quizzes and tests packaged with the textbook; these quizzes and tests were not
used during the course by the instructors. Items were selected by the third author, who was an
instructor in the Research Methods course, as providing good coverage of relevant material.
Some minor adaptations to improve the quality of distracters were made where needed. One
point was awarded for each correct response on all three measures, giving a potential minimum
Undergraduates’ Reading Success 12
for all three measures of 0; the potential maximum was 18 for the overall knowledge measure
and 10 on the topic knowledge measures. In our sample, the mean for overall knowledge was
12.56 (SD = 2.43), with a range of 7 to 18. The mean for topic knowledge of scientific approach
was 6.57 (SD = 1.95), with a range of 1 to 10. The mean for topic knowledge of statistical
inference was 4.49 (SD = 1.43), with a range of 2 to 8. A sample item for the overall knowledge
measure is: With ____, every member of the population has an equal probability of being
selected for the sample. a) simple random sampling; b) probability sampling; c) stratified random
sampling; d) random cluster sampling. A sample item for the topic knowledge measure for
scientific approach is: The fundamental characteristic of the scientific method is ____ a)
intuition; b) empiricism; c) replication; d) skepticism. A sample item for the topic knowledge
measure for statistical inference is: A Type I error occurs when the null hypothesis is ____ a)
rejected and the research hypothesis is true; b) accepted but the research hypothesis is actually
true; c) rejected but the null hypothesis is actually true; d) accepted but the null hypothesis is
actually false.
Reliabilities here presented a mixed and interesting picture. Using the standard of interitem consistency provided by Cronbach’s alpha, we calculated an α of 0.44 for the domain
knowledge assessment, 0.55 for the scientific approach topic knowledge assessment, and 0.017
for the statistical inference topic knowledge assessment. However, we would argue that for our
low-knowledge participants, the expectation that their knowledge (or lack thereof) will look like
a unitary and internally consistent construct when assessed in this situation is not necessarily
appropriate. They are likely to have fragmentary knowledge, which will present an inconsistent
picture in terms of scores on individual items. In particular, we did hear quite a few students
comment to the effect that they had taken a statistics course or courses, from which they appear
Undergraduates’ Reading Success 13
to have gleaned just such fragmentary and fragile knowledge related to the topic of statistical
inference. We did see greatest internal consistency for the assessment for which students could
be expected to have the most consistent knowledge base (scientific approach, which had been
covered early in the semester).
In addition, the homogeneity of the sample in regard to topic knowledge may have also
contributed to these low alphas, as Bernardi (1994) suggests. This may particularly be the case in
regard to the statistical inference topic knowledge test, for which the reduction in the response
variance (the average participant got fewer than half of the items correct, M = 4.49, SD = 1.43),
undoubtedly contributed to decreasing the alpha.
Nonetheless, we feel that the knowledge measures we used provide useful information
regarding how much they do know in such a fragmentary way on these subjects. A test-retest
approach to measuring reliability was not an option here. In addition, the think-alouds provided
considerable evidence that students were not responding in a random or inattentive way to the
questions on the knowledge assessments. We saw many instances of students making comments
connecting material they were reading to the material on the assessments, and identifying the
answer they had given earlier as right or wrong based on what they now had learned. Ultimately,
the reliability of an assessment should be considered in relation to its intended purposes (Burton,
2004). Here we are seeking to relate low-knowledge students’ relative success in terms of their
ability to recognize more or fewer correct answers associated with the information covered in the
given passages both to their activities while reading and to their ability to identify and restate
accurately from memory more or fewer (or any) important ideas from the passages they read. We
would argue that our measure has adequate reliability for this purpose, although we are
somewhat at a loss for a conventional way to quantify or evaluate such reliability.
Undergraduates’ Reading Success 14
Text Passages. The reading task used in this study involved two passages taken from the
course textbook (Cozby, 2007) that were selected and adapted to be as parallel as possible. One
passage, on the scientific approach (SA), was taken from the first chapter of the book and
presented material that should have been relatively familiar to the students as having been
covered in the assigned reading and discussed in class. The second passage, on statistical
inference (SI), was taken from chapter thirteen of the book, and presented material that was
expected to be less familiar to the participants. It had not yet been covered in class, but was
going to be covered in the classes immediately following students’ participation in this research,
so that it was not taken out of sequence in the coursework or presented before they should have
been adequately prepared to read it. Based on pilot data (Fox et al., 2008), this passage
addressing statistical inference was expected to present greater challenges in terms of
participants’ lower topic interest and knowledge, and in terms of the text’s lower perceived
familiarity, accessibility, and interestingness, due to the nature of its subject matter. Despite the
fact that it did not include any mathematical content, our pilot participants tended to see this
chapter as more mathematical in nature, and therefore brought to bear any predispositions or
preconceptions regarding their own likely interest or competence that mathematics evoked.
The selected passages were condensed slightly to provide a self-contained text that would
offer similar levels of difficulty for readers in terms of length, structure, and reading level. The
adapted passages were very close in length and difficulty (1670 words for scientific approach
and 1673 for statistical inference, Flesch-Kincaid Grade level of 12.0 for both, and FleschKincaid Reading Ease of 37.8 for scientific approach, 39.8 for statistical inference.) Each was
three pages of single-spaced text. Each included a title, headings, and bolded words, and each
had two figures, which were text boxes for the scientific approach passage and decision matrices
Undergraduates’ Reading Success 15
for the statistical inference passage. Figures were positioned (as they had been in the textbook) in
the center of the page.
Each text discussed a particular way of knowing about human behavior. The first passage
described the scientific approach, contrasting it to intuition and authority, and then going on to
discuss how scientists develop knowledge, the role of empiricism, and skepticism. The second
passage described how random error affects our ability to make conclusions about the true state
of affairs in the population, discussed the role of the null and research hypothesis, and the two
types of error in statistical inference. Each passage included appeals to the reader’s personal
experience and concluded with an extension of the discussion to everyday content and then a
very specific example of that everyday content. In the scientific approach passage, this involved
pseudoscientific claims and rumors presented in the media, and specifically a rumor regarding
the elimination of the gene for blond hair. In the statistical inference passage, the everyday
content related to type one and two errors as applied to jurors deciding on guilt or innocence and
then specifically to the situation of trying to decide whether someone is the right person to
marry.
Think-alouds. Participants were asked to think aloud while reading each of these
passages, and their think-aloud was recorded on an audiotape. They were told to verbalize what
they were thinking and doing as they read each passage, and practiced thinking aloud on a short
passage before moving on to the textbook passages. They could read aloud or not, as they chose.
They were told that when they finished reading the passage, they would be asked to respond to
open-ended questions regarding what they remembered and what they thought about the passage.
The 120 think-alouds (two were missing due to taping mishaps) were transcribed into text
documents by the first author and coded for strategic and evaluative/monitoring behaviors using
Undergraduates’ Reading Success 16
a set of 24 codes developed in two preceding pilot studies (Fox et al., 2007, 2008) and based on
the set of possible behaviors seen in verbal protocols of reading in Pressley and Afflerbach’s
summative overview (1995). The initial set of codes was expanded to better fit what seen from
these data, adding codes for rehearsing, expressing amusement, expressing surprise, arguing with
the text, and monitoring task completion status, all of which were also drawn from Pressley and
Afflerbach’s catalogue. The code for interpretation/elaboration was also split into two distinct
codes. This gave a total of 30 possible codes. The first and second authors independently coded
one-fifth of the transcripts (12 pairs of think-aloud transcripts for 12 participants), using this set
of 30 codes and a rubric developed by the first author. The list of codes and examples of coded
comments appear in Appendix A. Their exact agreement for assignment of codes was 91%
across 691 codes, with the differences resolved by discussion. This level of inter-rater reliability
was considered to be acceptable, and the first author coded the remainder of the think-aloud
transcripts using this coding scheme. The entry of these codes into a spreadsheet has not yet been
completed.
Learning outcomes. Learning outcomes in this study were assessed by an open-ended
question completed from memory after reading each passage: Please summarize this passage,
giving the main idea and as much supporting information as you can remember. Participants also
responded to an open-ended question involving application of the ideas presented in the passage;
we have not yet analyzed those data, and will not be reporting on them here. In addition,
participants were invited to tell us what they thought about how the author had presented the
material. They were given a full blank sheet for each question.
All learning outcomes were transcribed into text documents by the first author. They
were scored using a rubric in which one point was given for each correct statement of an
Undergraduates’ Reading Success 17
important idea. No deductions were made for incorrect statements. An example of a statement
coded as a correct expression of an important idea for the first passage (on the scientific
approach) is: “It is important to consider things like the person’s credibility, the institution they
represent, and where they receive their funding (so they are not biased.).” An example of a
statement coded as a correct expression of an important idea for the second passage (on
statistical inference) is: “The null hypothesis states that the independent variable had no effect on
the dependent variable.” One-fifth of the responses (12 pairs of outcomes for 12 participants)
were coded independently by the first and third author, with 88% agreement in assignment of
scores (and 96% agreement in identification of important ideas across all propositions
expressed). The three differences were resolved by discussion. This was considered to be an
acceptable level of reliability for this scoring scheme, and the first author then coded the
remainder of the outcomes.
Text evaluations. The evaluation comments produced by students in response to the third
outcome question for each passage were analyzed by the first author, who developed categories
under which to group the comments. Initial finer-grained distinctions were refined and regrouped
until a relatively stable set of seven broad categories emerged for their evaluations of the
presentation of the text material: accessibility (positive or negative), interestingness (positive or
negative), familiarity (positive or negative), text quality (positive or negative), the quality of the
text as an argument (positive or negative), author-related comments, and description of reading
strategies. Appendix B gives examples of comments falling into each of these categories. An
eighth code to allow identification of comparisons of the two passages was also included. The
first and third author independently coded one-fifth of the comment sets (12 pairs of comment
sets for 12 participants) using these eight codes according to a rubric developed by the first
Undergraduates’ Reading Success 18
author, with 90% exact agreement on assignment of codes across 173 comments. The differences
were resolved by discussion. This level of inter-rater reliability was considered to be acceptable,
and the first author used this scoring system for the remainder of the comment sets.
Procedure
Participants were told that their participation would involve two components. In the first
part, participants were given, in order, the demographics questionnaire, interest questionnaire,
need for cognition scale, domain knowledge measure, and topic knowledge measures.
Administration of the topic knowledge measures was counterbalanced such that 29
undergraduates saw the scientific approach measure first, while the other 32 saw statistical
inference first. There were no time limits for completion. Participants were encouraged to
respond to every item on each measure, and for the knowledge measures, were told that guessing
was okay. The average time for completing the first part was roughly 30 minutes. They could
then take a brief break before beginning the second part.
In the second part, participants practiced thinking-aloud with a short passage on
mosquitoes, from a popularly written science article by Marston Bates (1975) originally
published in Natural History magazine. This passage was intended to be mildly interesting but
not too challenging, so that participants might naturally find themselves responding to the topic
and descriptions. Participants varied considerably in their comfort with thinking aloud. Once
participants felt they were ready, they then read and thought aloud for either the scientific
approach or statistical inference passage. Administration of passages was counterbalanced such
that those 29 participants who saw the scientific approach topic knowledge measure first also
read that passage first, and the same for statistical inference.
Undergraduates’ Reading Success 19
When participants indicated that they were finished reading the passage, the passage was
taken away, and they were given the packet of three outcome questions. They were told that they
could complete the questions in any order and look back and forth between their responses on the
questions if they wished. Upon completing the outcome questions, the participants were again
reminded of the think-aloud directions and given the other passage, following the same
procedure as outlined above. There was no time limit for any of these tasks. Participants tended
to take very nearly the same time for each passage, and the average time for the second part was
roughly 60 minutes.
Results and Discussion
Observed differences across passages
Person-level. Reported pre-test interest for both the topic of scientific method (M = 2.85,
SD = 1.01) and for the topic of statistical inference and analysis (M = 2.90, SD = 1.34) was on
the negative side of the scale, toward “Not at all interested.” Analysis by means of t-tests showed
that the mean for each was significantly lower than interest in the other six listed topics, which
together had an average interest rating of 3.53 (p < .00). Pre-test knowledge of statistical
inference (M = 4.49, SD = 1.43) was significantly lower than knowledge of the scientific
approach (M = 6.57, SD = 1.95) when analyzed using a within-subjects repeated measures
ANOVA, F(1,60) = 55.74, p < .00.
Text-level. Overall, participants’ evaluative comments indicated that they were more
likely to find the SA passage to be accessible, interesting, and familiar than the SI passage,
although they found the quality of the texts to be comparable. About half of participants’
comments for each passage concerned accessibility, but for the SA passage these comments were
much more likely to be positive (40.3% of total comments for the passage) than negative (9.0%
Undergraduates’ Reading Success 20
of total comments), while for the SI passage they were more evenly split between positive
(30.3%) and negative (28.9%). Participants evaluated the SA passage positively for interest
(8.6%) slightly more often than they evaluated it negatively for interest (6.9%), while for the SI
passage their negative comments regarding interest (10.1%) were quite a bit more frequent than
their positive comments (2.7%). Only a few of their comments mentioned familiarity, but for the
SA passage those comments were all positive (3.4%), while for the SI passage negative
comments (2.7%) slightly outweighed the positive ones (1.8%). Text quality was evaluated
generally positively for both passages, with 16.7% of the comments for the SA passage being
positive evaluations of text quality, and 14.2% for the SI passage. Table 2 gives the breakdown
of evaluative comments by direction and category for both passages. In all but one of the 19
instances where participants directly compared passages, the comparison favored the passage on
scientific approach. Particular aspects of accessibility mentioned included organization, clarity,
coherence, flow of topics, text features, figures, examples, and real-life connections.
Interestingly, repetition was mentioned both as a positive and a negative aspect of accessibility;
participants were also attuned to the possible difficulties presented by vocabulary, which was
mentioned both as a stumbling block and as having been well-addressed in the passage.
Processing. In participants’ think-alouds, observed trends across passages included that
participants tended to evaluate agreement more often for the SA passage, and to identify
comprehension difficulties more often for the SI passage. For both passages, the most common
strategies used to address comprehension failures and solidify text understandings were
rereading and paraphrasing information at a local level. As is typical with think-aloud studies,
there was great variation in what was seen from participants in terms of types and levels of
activities (Pressley & Afflerbach, 1995). In this study, because we gave participants the choice to
Undergraduates’ Reading Success 21
read aloud or not, and also to begin to read aloud or to stop reading aloud at any point should
they choose, it appears that there may be some interesting patterns in terms of choices of when
and whether to read aloud. Verification of these trends and identification of further patterns will
be possible once the think-aloud codes are entered into a spreadsheet to enable a summative and
comparative view within and across participants that will support further quantitative and
qualitative analysis of the processing data.
Outcomes. Outcome scores for the SI passage (M = .77, SD = 1.13) were significantly
lower than those for the SA passage (M = 1.56, SD = 1.49) when analyzed using a withinsubjects repeated measures ANOVA, F(1,60) = 15.93, p < .00. More than half of the
participants (37) could not express accurately even one important idea from the SI text. In their
summaries for both passages, participants quite often merely listed the topics that had been
covered or the examples given rather than stating the important ideas presented in the course of
discussing these topics or examples. They were also quite likely to state important ideas that had
been presented, but to state them inaccurately. No participant failed to produce at least one
statement in response to the outcome question for each passage; they were actually trying to
answer it. We can reasonably expect that our undergraduate participants, predominantly
upperclassmen, were aware that a summary involves identification of important ideas, and that
where they failed to produce important ideas in their summaries it was because of deficiencies in
their mental representations of the text rather than a failure of interpretation of the outcome task
(Garner, 1985).
Participants could have inferred easily that they would be expected to produce some kind
of summary in which they identified important ideas when reading the second passage they saw,
so if the lack of specificity in the initial reading instructions (i.e., that they were not told
Undergraduates’ Reading Success 22
specifically to study the text, but only that we would be asking them about what they
remembered and what they thought about the text) were a factor in poor performance here, we
should have seen an improvement in outcome performance on the second passage, but we did
not.
We also were concerned that fatigue might be a factor affecting performance on the
second passage participants saw, although the counterbalanced order of administration of the
passages would keep such an effect from providing a competing explanation for our overall
findings. We found that the 29 participants who saw the SA passage first had a mean outcome
score of 1.66, slightly higher than those who saw it second, who had a mean outcome score of
1.47. However, the 32 participants who saw the SI passage first had nearly identical mean
outcome scores (.75) to those who saw it second (.76). No significant order effects were
detected.
Associations of variables
Quantitative patterns. Statistically significant correlations among our variables included
the moderate positive association of Need for Cognition (NFC) and total pre-test interest score (r
= .39). Total research methods knowledge was moderately positively correlated with both SA
topic knowledge (r = .39) and with SI topic knowledge (r = .37). SA topic knowledge showed a
moderate positive association with SA passage outcomes (r = .28). SA and SI passage outcomes
were also moderately positively correlated with each other (r = .33). NFC did not appear to be
related to either prior knowledge or outcomes as we measured them in our sample; NFC was also
not related to GPA. Correlations are given in Table 3. Although SI topic knowledge scores and
SI passage outcome scores were not significantly correlated (r = .14), it did appear that those
participants with the highest SI outcome scores (3 or 4) tended to have performed somewhat
Undergraduates’ Reading Success 23
better on the SI topic knowledge test (mean score of 5.38) compared to those whose outcome
scores were lower, who tended to have gotten fewer than half of the answers correct on the topic
knowledge test. Participants with an outcome score of 2 on the SI passage had a mean
knowledge score of 4.17, those with an outcome score of 1 on the SI passage had a mean
knowledge score of 4.30, and those scoring 0 on the SI passage had a mean knowledge score of
4.40.
The very low variability and restricted range in the outcome scores, particularly for the SI
passage, made it difficult to detect patterns of interactions of factors related to those outcome
scores using inferential statistical analysis. There were no significant interactions among our pretest variables of NFC, interest, and knowledge, in their associations with the outcome variables.
Qualitative patterns. To get another view of interactions of possible factors relating to
undergraduates’ success in reading, we attempted to map Alexander’s (2005) six reader profiles
onto our data using the pre-test information on topic knowledge and topic-related interest, the
think-aloud processing analysis, and also the outcome scores. The scheme that we used for this
coding is given in Appendix D. The first author did the initial coding of the profiles, and quickly
found that knowing about pre-test topic knowledge, topic interest, and on-line processing was
not adequate for identification of these profiles in our data. However, with the use of outcome
scores as well, good exemplars of the six profiles were found within our sample, including the
Challenged reader, which was somewhat unexpected in this pool of college students, many of
whom were on the verge of graduating. In addition, a potential additional profile was identified,
the Interest-Driven reader, whose engagement in reading depends on situational motivation as
sparked by interest in particular content during the course of reading. In many cases, however,
participants exhibited moderate levels of attributes that are framed as more extreme in the
Undergraduates’ Reading Success 24
profiles. A further difficulty was that even with observing the interaction of text and person level
factors across our two passages, we could not be sure we had sampled the full range of potential
behaviors; neither of our passages necessarily presented enough of a challenge to be sure that we
had tapped into how each of these reader types would respond to difficult texts, which is
necessary in order to obtain evidence of strategic processing across situations, particularly for the
Highly Competent Reader.
To get independent verification of these perceptions, the third author used the same
scoring scheme and independently scored a set of 14 profiles, selected by the first author to
include what were considered to be both clear exemplars of the seven profiles as well as cases
that were expected to present difficulty as far as assignment to a specific unique profile category.
He selected as clear exemplars the intended cases for the Interest Reliant, Non-Strategic,
Challenged, and Resistant readers. He also identified Knowledge-Reliant and Effortful as likely
assignments for the intended cases for those profiles, but saw these two profiles as presenting
rather similarly in these cases.. He did not identify the intended Highly Competent reader as
Highly Competent, seeing this participant as possibly either non-strategic or knowledge reliant.
In this case, he was not aware that an outcome score of 3 for the SI passage was relatively quite
high, so he did not consider this reader to have achieved a highly successful outcome on both
passages. And he did find all of the intended problematic cases to be problematic in terms of
matching them successfully to a profile.
This mapping of the reader profiles to our data was thus not entirely satisfactory. On the
one hand, we feel that the profiles have, in this situation, good explanatory value; where they do
fit, they give a satisfying explanation of what is seen in the way of a relationship between what
the reader comes in with, what they do in this situation, and what they are able to produce in the
Undergraduates’ Reading Success 25
way of learning outcomes. They have less predictive value in this situation due to the way in
which we generated them, because we needed to bring in outcome scores as part of our evidence
in determining which profile to assign.
A further difficulty with our use of the profiles lies in the presence in the profile
definitions of assumed knowledge regarding the reader’s feelings and knowledge about reading.
We do have some data on our participants as readers, in the form of their responses to the
reading habits question; although use of this data might help resolve some of the uncertainties for
some of the profiles, it still does not appear adequate to provide a decisive sorting mechanism.
Successful use of these profiles would require knowledge of participants’ knowledge
about reading (which might or might not equate to performance on an assessment of reading
ability), their interest in reading, their interest in the given subject matter, their knowledge of the
given subject matter, their typical level of engagement during reading, their level of engagement
while reading this particular text, how flexibly and appropriately they typically used reading
strategies to resolve detected difficulties in comprehension, how flexibly and appropriately they
used them in this situation, and at what level of processing (deep or surface) they were operating.
This brings us, in essence, back to the formulation by Waples, Berelson, and Bradshaw (1940),
particularly in the need to know readers’ typical behaviors; these are likely to be very difficult to
capture.
Investigation of factors relating to success in reading and learning from text requires
bringing together cross-situational aspects such as how these students are as readers (as in
Alexander’s profiles), how they approach learning from text in general (as in Marton & Säljö’s
deep and surface approaches), and also situational features including their knowledge and
interest in the topic, and their perception of the text. In addition, the difficulty of accurately
Undergraduates’ Reading Success 26
assessing these factors with appropriate sensitivity and grain-size is heightened in the type of
reading situation in which we are interested, where levels of knowledge and interest are likely to
be low or moderate, students are likely to be at similar novice levels, and the texts are from the
same textbook.
Further, there are still going to be readers who simply fall in-between, or who are not
consistent enough across situations to be definitively labeled as one reader type or another; there
will be Moderately Competent Readers, and Sometimes Knowledge-Reliant Readers. We remain
interested in knowing what person-level and text-level factors and interactions of these factors
are related to the reading success of these in-between or inconsistent reader types in a given
reading situation as well.
Case studies
Our final effort at detecting patterns of association and interaction in person-level and
text-level factors connected with relative success in reading in this situation was to undertake
case studies. Eight participants were selected for in-depth analysis based on their scores for NFC,
topic knowledge, outcomes, and processing behaviors. The rationale behind the selection of the
cases was that it was thought to be likely to be interesting and informative to take a much closer
look at what was going on where we saw unexpected patterns in the relation of prior knowledge
and outcome. A more detailed explanation of the selection process is given in Appendix C.
Expected patterns were those in which the outcomes appeared to be predictable based on
level of prior knowledge, that is, in which the participant had relatively high knowledge and was
relatively successful in outcome performance compared to other participants or those in which
they had relatively low knowledge and were relatively unsuccessful. A further expected pattern,
given that there were no consequences for lack of effort that resulted in low performance, was
Undergraduates’ Reading Success 27
one in which they had uniformly low outcomes across both passages regardless of their level of
knowledge, although we recognize that there could very well be differences in level of effort,
and that uniformly low outcomes regardless of level of knowledge does not mean necessarily
that the participants were not trying. Unexpected patterns were those in which participants had
high scores with low knowledge for either passage or both passages, or a high score with high
knowledge for one passage, and a low score with high knowledge for the other, indicating that
something different was happening in the way of effort for the two passages. There were nine
such unexpected patterns, but one pattern was not populated, as is shown in Table 1 in Appendix
C. We chose our case studies from the remaining eight patterns, selecting where possible
participants who were at the high or low end of the distribution of NFC scores, and those whose
think-aloud transcripts revealed marked differences in processing across the two passages.
Case 1 (P 3) – Low outcome with high knowledge (SI). This participant was a 20-year old
female with 85 cumulative college credits and a self-reported GPA of 3.0. Her total NFC score of
56 put her in the bottom third of the distribution for our sample. She had a relatively high
knowledge score of 7 out of 10 for the scientific approach pretest and a relatively high outcome
score of 2 for her summary of important ideas for this passage. Her knowledge score was also
relatively high for the statistical inference pretest, 5 out of 10, while her outcome score for this
passage was 0. She reported a neutral pre-test level of interest (3) in the topic of the scientific
approach passage and a moderately positive level of interest (4) in the topic of the statistical
inference passage. In her post-test evaluative comments, she noted that the scientific approach
passage was easy to follow and that the writing was not too complicated, although she had to reread some long sentences. For the statistical inference passage, she commented that she had had
Undergraduates’ Reading Success 28
a lot of difficulty in understanding it, probably because of the difficulty she has in thinking about
scientific terms, although she liked the use of a real-life situation at the end.
While reading, she worked extremely hard at both passages. The catalogue of strategies
she brought to bear for each is long, including underlining, connecting to prior knowledge,
paraphrasing, and interpreting. An immediately obvious difference in her behaviors between the
two passages is that she read the first passage (scientific approach) aloud, while for the second
passage (statistical inference) she read portions and skimmed and paraphrased portions. Beyond
this somewhat surface difference in behaviors, she appeared to take a markedly different
approach to the task involved between the two passages; this may have been due in some part to
her awareness for the second passage (on statistical inference) that she would be asked to recall
the main ideas.
She used a very methodical, line-by-line approach to reading, building an understanding
phrase by phrase and sentence by sentence; she frequently substituted restatements for the given
language, and connected back to earlier statements in the text. However, for the material
presented in the statistical inference passage, she fixated on specific pieces of information in a
way that hindered her from getting a larger understanding of what the passage was saying. She
rehearsed and repeated these, going back to them several times in an effect to fix them in her
memory. She commented that she was very bad at understanding terms like this, and referred to
her level of readiness for the outcome task as she came to the end of the passage. She seemed to
have moved her focus away from understanding the material, as in the first passage, toward
identification and retention of important information, although her level of effort and
engagement was equally high. Related to this, she ended up being unable to state correctly what
any important ideas of the second passage had been.
Undergraduates’ Reading Success 29
Case 2 (P 42) – Low outcome with high knowledge (SA). This participant was an 18-year
old male with 15 cumulative college credits and a self-reported GPA of 2.8. His total NFC score
of 69 put him in the top third of the distribution for our sample. He had a very high knowledge
score of 9 out of 10 for the scientific approach pretest and a relatively low outcome score of 1 for
his summary of important ideas for this passage, which he saw second. His knowledge score was
also relatively high for the statistical inference pretest, 5 out of 10, while his outcome score for
this passage was 1, in the top half of the distribution. He reported a moderately positive level of
interest (4) in the topics of both passages. In his post-test evaluation comments, he commented
that the scientific approach passage was interesting, familiar and concise, but that he didn’t think
the text boxes helped much to organize the information. For the statistical inference approach, he
noted that certain terms seemed important to remember for a test, and that the examples at the
end seemed memorable.
This participant also worked through both texts line by line; he seemed to set himself the
task of reporting on his thoughts after nearly each proposition or clause in the text. His level of
response thus was high for both passages, but the content of the response differed markedly
between the two passages. For the statistical inference passage, his efforts seemed directed more
to building an understanding of this relatively unfamiliar material; he worked on interpreting,
connecting to prior text, paraphrasing, and connecting to relevant prior knowledge and
experience. For the scientific approach passage, he did not seem to need to work at
understanding. As his topic knowledge score indicates and as he noted in his evaluative
comments, he was familiar with this material. As a consequence, he devoted more of his
comments to evaluating agreement with the material, arguing with the text, elaborating and
connecting to personal experience in ways that were not particularly relevant or helpful. Where
Undergraduates’ Reading Success 30
the text presented a challenge, as in the first passage, he was quite ready and able to engage in
effortful reading and understanding, which was associated with a relatively high outcome score.
Where it did not, he turned the focus of his engagement elsewhere. Related to this, his outcome
performance was relatively low.
Case 3 (P 31) Low outcome with high knowledge (SA), High outcome with low
knowledge (SI). This participant was a 22-year old female with 106 cumulative college credits
and a self-reported GPA of 3.6. Her total NFC score of 82 put her at the very top of the
distribution for our sample. She had a high knowledge score of 8 out of 10 on the scientific
approach pretest and a relatively low outcome score of 1 for her summary of important ideas for
this passage. Her knowledge score was low for the statistical inference pretest, 2 out of 10, while
her outcome score for this passage, which she saw second, was 1, a relatively high outcome
score. She reported a moderately positive interest (4) in the topic of the scientific approach
passage and a strongly positive interest (5) in the topic of the statistical inference passage. In her
post-test evaluation comments, she commented about each passage that it was organized. For the
scientific approach passage, she noted that the headings and figures were helpful, while for the
statistical inference passage, she mentioned that she had been concerned because she had last
encountered such material over a year ago, and that she felt she had adopted of the author’s use
of organization to help her recall ideas.
This participant moved relatively quickly through both passages, with no reading aloud
and little rereading. Her dominant behaviors while reading the scientific approach passage,
which she saw first, were interpreting, evaluating agreement, and connecting to personal
experience. Her very first comment was to note that this information was familiar to her, and that
she recalled discussing it in class. She did not appear to be building any kind of larger
Undergraduates’ Reading Success 31
understanding of the meaning of the passage, but cruised along, sampling from the ideas
presented in her evaluations of agreement and inferential extensions of the content. For the
statistical inference passage, on the other hand, she began by noting that she had a preconception
that she might not understand the material. Her dominant behaviors for this passage included
checking that she understood, and connecting to prior knowledge and personal experience. Her
rereading behaviors, although similar in frequency, differed in scope across the two passages; for
the first passage, she reread single sentences, while for the second passage she felt the need to
reread larger portions of material, going over entire paragraphs or explanations. She also noted
uncertainty about her answers on the pretest, and that she had not been able to remember the
content from having taken statistics before. At the end of the passage, she noted that she found
the example of the jury making a decision to be very helpful in solidifying her understanding of
the content. Her reading in this case appeared more effortful and focused on building an
integrated understanding of what was presented.
Case 4 (P 25) Low outcome with high knowledge (S1), High outcome with low knowledge
(SA) This participant was a 22-year old female with 80 cumulative college credits and a selfreported GPA of 2.0. Her total NFC score of 69 put her in the top third of the distribution for our
sample. She had a relatively low knowledge score of 6 out of 10 on the scientific approach
pretest and a relatively high outcome score of 4 for her summary of important ideas for this
passage. Her knowledge score was relatively high for the statistical inference pretest, 5 out of 10,
while her outcome score for this passage, which she saw first, was 0. She reported a neutral level
of interest (3) in the topic of the scientific approach passage and a strongly negative interest (1)
in the topic of the statistical inference passage.
Undergraduates’ Reading Success 32
This participant read both passages aloud in their entirety. She had what seemed to be a
difficult and frustrating experience reading the first passage, on statistical inference. She
frequently commented on her level of comprehension, and went back and reread when she did
not understand. She did not seem to have any particular strategies available when she did not
understand other than rereading. She also rehearsed information to try to store it in her memory.
Her evaluative post-test comments indicated that she found the passage confusing, and that she
would have liked more examples. She also noted that when she does not understand she has to
reread or try to make up her own examples (although she gave no evidence of doing so here). For
the scientific approach passage, on the other hand, she was operating on a different level, arguing
with the text and considering whether she agreed or disagreed with the author’s statements. She
spent less time monitoring her own comprehension, and much less time rereading. Her post-test
evaluative comments specifically stated her preference for this passage over the other, because it
was more interesting and related better to “people in ordinary use.” And she was much more
successful in recalling important content as a result of this reading experience than when she had
to struggle to comprehend by rereading.
Case 5 (P 20) High outcome with low knowledge (SI) This participant was a 23-year old
female with 72 cumulative college credits and a self-reported GPA of 3.0. Her total NFC score of
49 put her in the bottom third of the distribution for our sample. She had a low knowledge score
of 5 out of 10 on the scientific approach pretest and a relatively low outcome score of 1 for her
summary of important ideas for this passage. Her knowledge score was low for the statistical
inference pretest, 4 out of 10, while her outcome score for this passage, which she saw first, was
2, a relatively high outcome score. She reported a neutral level of interest (3) in the topics of both
passages.
Undergraduates’ Reading Success 33
This participant read both passages aloud in their entirety. While reading the statistical
inference passage, which she saw first, she commented several times that she was tired and that it
wasn’t interesting – she was working to understand it, and her dominant behavior next to rereading was detecting that she didn’t understand something. When she didn’t understand, she
often went back and re-read; this re-reading was occasionally followed by a positive evaluation
of comprehension. In addition to rereading, she also asked questions, connected to prior text, and
restated as she went along. At the end, she noted that it wasn’t particularly interesting, although
in her evaluative comments she stated that she thought the presentation was good because it was
detailed, and the examples related helpfully to practical problems. When reading the passage on
the scientific approach, her dominant behaviors were rereading, questioning and evaluating
agreement, although she also frequently monitoring her own level of comprehension, stating that
she didn’t understand or that she did. She expressed a high level of interest as she was reading
this passage, and her post-test evaluative comments noted that she found the passage to be
interesting and the author to be giving good support for his points. She seemed to process this
passage much more as a piece of persuasive text. Her response to the outcome question,
however, tended to focus on less important details; she had not successfully pulled out the gist of
the important ideas.
Case 6 (P 18) High outcome with low knowledge (SA) This participant was a 19-year old
female with 64 cumulative college credits and a self-reported GPA of 3.8. Her total NFC score of
66 put her just below the top third of the distribution for our sample. She had a relatively low
knowledge score of 6 out of 10 on the scientific approach pretest and a relatively high outcome
score of 3 for her summary of important ideas for this passage. Her knowledge score was low for
the statistical inference pretest, 3 out of 10, while her outcome score for this passage, which she
Undergraduates’ Reading Success 34
saw first, was 0. She reported a moderately negative interest (2) in the topic of the scientific
approach passage and a moderately positive interest (4) in the topic of the statistical inference
passage.
This participant saw the statistical inference passage first, and proceeded by skimming
through it, making interpretations, checking for understanding, and paraphrasing. She appeared
to be making a moderate effort to understand, but her interpretations and paraphrases were not
particularly faithful to the meaning of the text, and thus likely did not help her to construct a
good representation of its gist. Her response to the outcome question for this passage was merely
a list of topics that had been addressed. In her evaluative comments she noted that she found the
presentation of the material to be effective in its use of simple language, multiple forms of
description, real life situations, and visual cues.
For the scientific approach passage, she read the entire text aloud rather than skimming.
In addition, she attended closely to what struck her as important elements, underlining them and
repeating them aloud. Her other dominant activity was agreeing with the text, which she did not
do at all with the earlier passage. She offered many interpretations for this passage as well, but
here they tended to be accurate or reasonable interpretations. In her comments on this passage,
she noted that she thought the material was presented well in being easy to read and
understandable, in its use of common words and definitions for unfamiliar terms. However, she
also noted that she found it repetitious, that the material was just common sense and didn’t need
so much explanation, and that by the end she was bored. Nonetheless, she was able to connect
successfully to this commonsense material and identify a number of the important ideas that had
been presented.
Undergraduates’ Reading Success 35
Case 7 (P 24) High outcome with low knowledge (SI). This participant was a 21-year old
male with 65 cumulative college credits and a self-reported GPA of 2.3. His total NFC score of
47 put him near the bottom of the distribution for our sample. He had a high knowledge score of
10 out of 10 on the scientific approach pretest and a high outcome score of 6 for his summary of
important ideas for this passage. His knowledge score was low for the statistical inference
pretest, 3 out of 10, while his outcome score for this passage, which he saw first, was 2, a
relatively high outcome score. He reported a neutral level of interest (3) in the topic of the
scientific approach passage and a somewhat negative interest (2) in the topic of the statistical
inference passage.
This participant read each passage aloud in its entirety. He interrupted his reading much
more often to comment for the first passage, on statistical inference. He checked on his own
understanding, paraphrased both at a local and a global level, offered interpretations, evaluated
agreement, and was generally more active in his interaction. In his evaluative comments, he
noted his lack of familiarity with this material, and thought that he would need more of a
background in statistics to understand it better. He commented that he had to read slowly to
understand the concepts and had to think harder to understand some of the terminology. His
effort was reflected in a reasonably high level of success in identifying important ideas for this
passage.
For the scientific approach passage, his reading was much more streamlined. He read it
pretty much straight through, pausing only a few times; he offered an interpretation, made a
global restatement, connected to personal experience, and at the end, briefly looked back and
rehearsed important terms. He appeared to be going through this passage much more easily, in
Undergraduates’ Reading Success 36
the sense that much less overt strategic processing was necessary. His outcome score for this
passage was quite high, supporting this view of what he was doing as he read.
Case 8 (P 45) High outcome with low knowledge (SA, SI). This participant was a 21-year
old female with 104 cumulative college credits and a self-reported GPA of 3.2. Her total NFC
score of 45 put her near the bottom of the distribution for our sample. She had a low knowledge
score of 4 out of 10 on the scientific approach pretest and a relatively high outcome score of 3
for her summary of important ideas for this passage. Her knowledge score was low for the
statistical inference pretest, 2 out of 10, while her outcome score for this passage, which she saw
second was 1, a relatively high outcome score. She reported a moderately positive interest (4) in
the topic of the scientific approach passage and a strongly positive interest (5) in the topic of the
statistical inference passage.
She saw the scientific approach passage first, and looked through it quite quickly, not
reading aloud or rereading at all. Her few comments included connections to personal experience
and interpretations. She also argued with the text and commented on her agreement. Her
evaluative comments indicated that she found the passage to be informative and organized, and
that she found it helpful that the examples used were all familiar to her from her own experience.
In her summary, however, she did not merely list the examples, but was able to pull together
important themes from the passage. For the statistical inference passage, she did some reading
aloud, but no rereading. She paraphrased what she was reading quite a few times, which was not
a behavior seen for the first passage at all. Again, she offered interpretations, and here connected
to prior knowledge. her evaluative comments indicated that she found the passage to be well
organized and informative, but that it was not exciting, and that it was difficult to keep focused
on only definitions. She liked the examples at the end, and thought more examples would have
Undergraduates’ Reading Success 37
improved the presentation. Again, in her response, she did not mention the examples, but focused
on restating important ideas presented in the passage.
Summary
Among the findings that emerge from these case studies is the importance of readers’
perception of the task at hand; there appear to have been marked differences in how they
interpreted what was called for from them in reading these passages, possibly in some cases as a
result of differences in level of prior knowledge, possibly in some cases as a result of the
awareness of the nature of the outcome task, and possibly in some cases as a result of different
reactions to the topics of the passages and the way in which the passage content was presented.
The importance of having effective strategies available is also suggested, as is the importance of
attending to applying the strategy successfully; in particular, the possibly helpful or hindering
nature of connections to personal experience and prior knowledge is evident. Related to task
perception is the role of both level of effort and level of care in building an understanding of
what is read; successful readers did not necessarily expend the most effort, but they did attend
with care to the accuracy and integration of the understanding they built. The role of NFC in
motivating engagement is mixed in what we see here in these case studies. For both low and high
NFC participants, difficulty in understanding the text could promote engagement, while
perceived lack of challenge or lack of interest could lead to lower engagement or engagement at
a more superficial level, along with poorer learning. And, as in Case 4, difficulty in
understanding the text, along with perceived difficulty, unfamiliarity or lack of interest sparked
by the text could lead to confusion and disengagement, while a lack of effective reading
strategies could mean that what effort was expended was not helpful. For this participant, easier
and friendlier text promoted making connections, readier engagement, and building of studier
Undergraduates’ Reading Success 38
understandings. Appropriate level of challenge is clearly an important issue here, and rests
squarely on the interaction of the reader-level and text-level factors.
An interesting finding emerging clearly here in these case studies as well as in the thinkalouds overall was the tendency of some of the participants to approach these two passages as
being two very different text types, even though they were from the same textbook and by the
same author. They tended to address the passage on statistical inference as being detail-oriented
and much more factual, calling forth efforts to extract information and memorize it. The passage
on the scientific approach was treated as more argumentative or persuasive, evoking judgments
of agreement or disagreement. Participants also far more often read the material in the figures in
the scientific approach passage, which were text boxes. Many of them skipped right over the
decision matrices in the statistical inference passage, which were not in the same way part of a
flow of text. Textbook design and content presentation are also clearly important; although
participants felt the text quality of both passages to be good, overall, they reported the scientific
approach passage to be much more accessible and interesting. These undergraduates appeared to
be quite familiar with textbook features such as bolded words and headings; they were quite
sensitive to repetition and redundancy in the explanations, to the use of familiar and relevant
examples, to care in defining unfamiliar terms, and even to length of sentences and paragraphs.
However, the same text aspects were perceived both positively and negatively by different
participants; one reader’s redundancy was another reader’s helpful repetition, for instance.
Perception of the text, too, appears to be a matter of interaction of reader-level and text-level
factors.
Research and Educational Implications
Undergraduates’ Reading Success 39
We highlight here the importance of attention to activity, person and task factors in trying
to understanding undergraduates’ differences in success in learning from text. Isolating a single
factor or even a few factors is not likely to support an adequate account. Although Alexander’s
reader profiles offer a multidimensional analytic tool, we identified here some difficulties in
applying this tool. We also saw that Need for Cognition alone was not a strong explanatory
factor; investigation of alternative formulations and measurements of reading approach and
engagement may be necessary.
We found that undergraduates often were not particularly successful readers in the
situation we offered them, and are concerned that even in situations offering a stronger role for
performance goal orientations, they might yet fall short. They should be able to learn from text in
such a situation; if they cannot, they are hampered as independent learners. There were clearly
strong differences between students in what they had access to in the way of appropriate and
helpful reading strategies, reading behaviors, and task perceptions in this situation. Finally,
attention to helping students develop motivational self-regulation may be needed; passively
waiting for the text or situation to make them be interested enough to learn is not a successful
approach to reading or to learning in general.
Undergraduates’ Reading Success 40
References
Alexander, P. A. (2005). The path to competence: A lifespan developmental perspective on
reading. [White paper commissioned by the National Reading Conference]. Oak Creek,
WI: National Reading Conference.
http://www.nrconline.org/publications/ThePathToCompetence.pdf
Alexander, P. A., & Jetton, T. (2000). Learning from text : A multidimensional and
developmental perspective. In M. L. Kamil., P. B. Mosenthal, P. D. Pearson, & R. Barr
(Eds.), Handbook of reading research: Volume III (pp. 285-310). Mahwah, NJ: Erlbaum.
Armbruster, B. B. (1984). The problem of “inconsiderate” text. In G. G. Duffy, L. R. Roehler, &
J. Mason (Eds.), Theoretical issues in reading comprehension (pp. 202-217). New York:
Longman.
Bates, M. (1975). The lady lives on blood. In A. Ternes (Ed.), Ants, Indians, and little dinosaurs
(pp. 74-82). New York: Charles Scribner’s Sons.
Bernardi, R. A. (1994). Validating research results when Cronbach's alpha is below .70: A
methodological procedure. Educational and Psychological Measurement, 54, 766-775.
Burton, R. F. (2004). Multiple choice and true/false tests: Reliability measures and some
implications of negative marking. Assessment & Evaluation in Higher Education, 29,
585-595.
Cacioppo, J. T., Petty, R., E., Feinstein, J. A., & Jarvis, W. B. G. (1996). Dispositional
differences in cognitive motivation: The life and times of individuals varying in need for
cognition. Psychological Bulletin, 119, 197-253.
Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition.
Journal of Personality Assessment, 48, 306-307.
Undergraduates’ Reading Success 41
Cozby, P. C. (2007). Methods in behavioral research (9th ed.). New York: McGraw Hill.
Dai, D. Y., & Wang, X. (2007). The role of need for cognition and reader beliefs in text
comprehension and interest development. Contemporary Educational Psychology, 32,
332-347.
Ferstl, E. C., & Kintsch, W. (1999). Learning from text: Structural knowledge assessment in the
study of discourse comprehension. In H. van Oostendorp & S. R. Goldman (Eds.), The
construction of mental representations during reading (pp. 247-277). Mahwah, NJ:
Lawrence Erlbaum Associates.
Fox, E., Dinsmore, D. L., & Alexander, P. A. (2007, April). Situational success at reading
challenging texts: Exposing the fragile understanding of college students. In P. Alexander
(Chair). Fragile understanding: When good ideas go bad. Symposium presented at the
annual meeting of the American Educational Research Association, Chicago, IL.
Fox, E., Dinsmore, D. L., Maggioni, L., & Alexander, P. A. (2008, March). Undergraduates’
independent and scaffolded reading of course texts: Further evidence of fragile
understanding. Paper presented at the annual meeting of the American Educational
Research Association, New York.
Garner, R. (1985). Text summarization deficiencies among older students: Awareness or
production ability? American Educational Research Journal, 22(3-4), 299-312.
Kintsch, W. (1980). Learning from text, levels of comprehension, or: Why anyone would read a
story anyway. Poetics, 9, 87-98.
Marton, F., & Säljö, R. (1997). Approaches to learning. In F. Marton, D. Hounsell, & N.
Entwisle (Eds.), The experience of learning (2nd ed., pp. 39-58). Edinburgh: Scottish
Academic Press.
Undergraduates’ Reading Success 42
McKeown, M. G., Beck, I. L., Sinatra, G. M., & Loxterman, J. A. (1992). The contribution of
prior knowledge and coherent text to comprehension. Reading Research Quarterly, 27,
79-93.
Paris, S. G., Wasik, B. A., & Turner, J. C. (1991). The development of strategic readers. In R.
Barr, M. L. Kamil, P. Mosenthal, & P. D. Pearson (Eds.), Handbook of reading research:
Volume II (pp. 609-690). Mahwah, NJ: Lawrence Erlbaum Associates.
RAND Reading Study Group (2002). Reading for understanding: Toward an R&D program in
reading comprehension. Santa Monica, CA: RAND.
Roberts, J. C., & Roberts, K. A. (2008). Deep reading, cost/benefit, and the construction of
meaning: Enhancing reading comprehension and deep learning in sociology courses.
Teaching Sociology, 36, 2008, 125-140.
Säljö, R. (1997). Reading and everyday conceptions of knowledge. In F. Marton, D. Hounsell, &
N. Entwisle (Eds.), The experience of learning (2nd ed., pp. 89-105). Edinburgh: Scottish
Academic Press.
Schiefele, U. (1999). Interest and learning from text. Scientific Studies of Reading, 3, 257-279.
Simpson, M. L., & Nist, S. L. (2002). Encouraging active reading at the college level. In C. C.
Block & M. Pressley (Eds.), Comprehension instruction: Research-based best practices
(pp. 365-379). New York: Guilford Press.
Wade, S. E., Buxton, W. M., & Kelly, M. (1999). Using think-alouds to examine reader-text
interest. Reading Research Quarterly, 34, 194-216.
Waples, D., Berelson, B., & Bradshaw, F. R. (1940). What reading does to people. Chicago, IL:
University of Chicago Press.
Undergraduates’ Reading Success 43
Appendix A
Codes for Think-Alouds
Strategic behaviors

Reading aloud

Re-reading

Adjusting reading rate when re-reading – speeding up or slowing down

Skimming (reading aloud while skipping portions)

Guessing the meaning of a word in context [“Erroneous I think means things that are not
necessarily factual.”]

Predicting [“Okay, now it’s going to summarize that.”]

Questioning [“What would happen if you do it either direction?”]

Arguing with text [“it also, it really depends on your knowledge of the subject, ‘cause if
you don’t know much about it, it won’t seem vague or improbable evidence.”]

Underlining or other marking on the text [“Underlining intuition and authority.”]

Using text feature [“I’m looking for it in the table.”]

Rehearsing (repeating information to maintain it in memory) [“So that’s type one error.
Type two. Okay. Type one, type two.”]

Restating (paraphrase) or repeating text information
-
local (word, phrase, sentence level) [“So significance level can increase or decrease
the type one error.”]
-
global (paragraph, passage level) [“So basically it introduces about, um, how the
scientific approach differs from just, uh, intuition and authority.”]

Making connections
Undergraduates’ Reading Success 44
-
to background knowledge [“We learned about peer review in class, it’s when other
people kind of look at your results and confirm it.”]
-
to personal experience [“That happened to my sister.”]
-
to prior text [“Intuition and authority are the things I just read about.”]
-
to topic knowledge test [“This probably relates to scientific skepticism, which was the
thing, a question on one of those tests I just took.”]

to research task [“I’m not gonna have much to write about this.”]
Interpreting (a statement requiring reasoning beyond information in the text to build text
meaning) [“So, that’s just talking about the confidence interval.”]

Elaborating (a statement requiring the use of additional information not explicitly in the
text to build beyond text meaning or pursue a non-text related train of thought) [“what if,
um, what if that one person just, like, stole a biscuit or something.”]
Monitoring/Evaluative behaviors

Evaluating comprehension (positive or negative) [“I’m already confused by this
passage.”]

Evaluating agreement with text (positive or negative) [“That’s definitely true.”]

Evaluating text quality [“That’s a good way to describe it.”]

Evaluation of interest (positive or negative) [“The first part was kind of interesting.”]

Evaluation of importance of text [“I feel like that’s important, with the, to know for later
on this semester.”]

Evaluation of task difficulty [“In order for me to really realize what is going on here, I
would have to sit down and study this stuff.”]

Monitoring task completion status [“Okay, I’m done.”]
Undergraduates’ Reading Success 45
Other

Expression of empathy (sympathy or feelings felt or imputed to others) [“That’s, that’s
really nice when people adopt children.”]

Expression of amusement [“Um, [laughs] I was thinking it was funny.”]

Expression of surprise [“Surprised by the findings.”]

No code (not enough information available to determine a code, as when comments are
partially unintelligible or fragmentary) [… Okay, so that’s [unintelligible] intuition and
Aristotle.”]
Undergraduates’ Reading Success 46
Appendix B
Codes for Evaluative Comments
Accessibility

Positive [“I thought the author’s presentation of the material was easy to understand.”]

Negative [“complex terms & abstract ideas”]
Familiarity

Positive [“In fact, it was very similar to the textbook representation of Type I + Type II
errors + null + alternative hypotheses that I had several years ago in a statistics class.”]

Negative [“Unfortunately majority of information was new.”]
Interest

Positive [“and had my interest almost the entire time.”]

Negative [“but not very intriguing.”]
Text quality

Positive [“Overall, the material was well presented.”]

Negative [“I think the author presented the information poorly.”]
Argument

Positive [“and I actually did not see any contradiction of what the author was saying.]

Negative [“Rather than a presentation of facts, the author seemed to want to convince the
reader of the importance of the scientific method.”]
Author-related

[“The author seems to be very devoted to the subject matter, very impassioned.”]
Reading Strategy

[“Thinking back, I tried to implement his use of organization to recall ideas.”]
Undergraduates’ Reading Success 47
Comparison

[“This article was better than the first.”]
Undergraduates’ Reading Success 48
Appendix C
Rationale Behind the Eight Cases Chosen for Case Study
Grouping of Participants
Participants were grouped into categories based on their pre-test performance on the topic
knowledge test for each topic, their outcome scores for each passage, and their NFC scores. High
and low assignments were determined as follows.
For knowledge and outcome measures this was (roughly) a median split:
High for Knowledge – Scientific Approach was a score from 7-10 (top 34)
Low for Knowledge – Scientific Approach was a score from 1-6 (bottom 27)
High for Outcome – Scientific Approach was a score from 2-6 (top 27)
Low for Outcome – Scientific Approach was a score from 0-1 (bottom 34)
High for Knowledge – Statistical Inference was a score from 5-8 (top 30)
Low for Knowledge – Statistical Inference was a score from 2-4 (bottom 31)
High for Outcome – Statistical Inference was a score from 1-4 (top 24)
Low for Outcome – Statistical Inference was a score of 0 (bottom 37)
For need for cognition, this was the top and bottom third of the distribution.
High for Need for Cognition was a score from 67-82 (top 21)
Low for Need for Cognition was a score from 41-57 (bottom 21)
(Participant 53 did not complete the Need for Cognition questionnaire.)
Enumeration of the Possible Arrangements of Knowledge and Outcome Assignments
There are sixteen possible combinations, as shown in Table 1. Of these, seven are
predictable patterns, which include consistent associations of high/low knowledge and outcome
for each passage or consistently low outcome performance across passage regardless of
Undergraduates’ Reading Success 49
knowledge. In the nine unpredictable patterns, something appeared to be going on for the two
passages that was not just related to how much knowledge they came in with or to a general lack
of effort in this situation.
From the cases in the unexpected patterns, we pulled out eight to investigate more
closely, one for each populated pattern. We chose participants with high or low NFC scores over
those with mid-level scores, and participants whose think-aloud transcripts revealed differences
in behaviors between the two passages over those who were behaving more similarly. These two
criteria together enabled the clean identification of one good candidate in each of our eight
patterns.
Undergraduates’ Reading Success 50
Table 1. Expected and unexpected patterns of knowledge and outcome assignments
SA Knowledge
Expected
Unexpected
.
SA -
SI -
SI -
Outcome Knowledge Outcome
Participants
(High NFC) (Low NFC)
H
H
H
H
6, 8, 9, 47, 27, 39, 10, 53
H
H
L
L
17, 36, 48, 1
L
L
H
H
49, 54
L
L
L
L
13, 32, 59, 7, 29, 30, 22, 41, 57,
L
L
H
L
40, 50, 43, 60, 4
H
L
L
L
46, 35, 55, 28, 38, 44
H
L
H
L
37, 14, 16, 56
H
H
H
L
61, 3, 51, 19
H
L
H
H
42, 15, 52
H
H
L
H
12, 21, 24
L
H
H
H
H
L
L
H
31, 33
L
H
H
L
25, 26, 2, 5,
L
L
L
H
11, 20, 34, 58
L
H
L
L
18
L
H
L
H
45, 23
Undergraduates’ Reading Success 51
Appendix D
Coding Scheme for Reader Profiles
Available data:
Coded transcript as evidence of both types of behaviors and level of engagement
Summary table of frequency of codes for each passage
IQ Total = total interest score (possible range 8–40)
Int SA = interest in the scientific approach as a topic (possible range 1–5)
Int SI = interest in statistical inference as a topic (possible range 1–5)
DK Total = total domain knowledge score (possible range 0–18)
TK SA = total topic knowledge score for scientific approach (possible range 0–10)
TK SI = total topic knowledge score for statistical inference (possible range 0–10)
OUT SA = outcome score for scientific approach (number of important ideas correctly stated)
(range 0 – 6)
OUT SI = outcome score for statistical inference passage (number of important ideas correctly
stated) (range 0 – 4)
Expected relevant variables for each profile:
Effortful Processor: Either low or high background knowledge, not particularly effective
strategy use; however, high engagement in trying to get somewhere with the reading, and not
much control over the outcome, so could be low or high depending on the task difficulty /
background knowledge / how well the strategies they have happened to align with the problems
they were encountering. The hallmark of this one is hard work, but not terribly well-directed
effort. They would generally go line by line.
Undergraduates’ Reading Success 52
Knowledge Reliant: Knowledge reliant readers should do better on texts for which they have
higher prior knowledge; they should prefer to use the strategy of connecting to prior knowledge
or personal experience, and do better when they can make these connections in a helpful way.
There is a link for them between what they can get access to in terms of prior knowledge and
how successful they will be in making sense of a given text. So the key variables are prior
knowledge, use of connecting to personal knowledge stores as a strategy, and outcomes linked to
these. They might or might not show high engagement or interest.
Non-Strategic: Non-strategic readers do not have access to a helpful repertoire of strategies. As
soon as they run into difficulty understanding something, they are stuck, because they just do not
have anything helpful to do to help them out. You should see repetition of one or two strategies,
and tendency toward lack of successful resolution of comprehension difficulties when they
occur.
Highly Competent: These readers know a lot about reading, and tend to be in control of their
reading outcomes. For difficult texts, even those where they have little background knowledge or
low interest, they know how to bring into play effective strategies. They also know how to be in
control of their comprehension as it’s being constructed – they keep track of important points
and whether they are getting the big picture – they do not necessarily worry about understanding
every word or going line by line. So they should have good outcome scores, regardless of their
level of prior knowledge or interest.
Challenged: These readers have not mastered basic reading processes, so are stymied when
trying to construct meaning because they are having trouble at the word or sentence level. They
are not fluent readers, and struggle with reading aloud. Their outcome scores would be low no
matter what their level of knowledge or interest related to the topic, because of their low level of
Undergraduates’ Reading Success 53
knowledge of reading and their inability to get to a point of being able to use strategies to build
larger meaning. But they typically also lack background knowledge and interest, just because of
their inability to read successfully. They could have low engagement, but there should be some
evidence that they are struggling with understanding.
Resistant: These readers might be capable of reading effectively, but choose not to. They refuse
to engage appropriately. So they could have knowledge of the topic or of reading, and be quite
capable of using strategies, but do not. So their outcome scores are low, their engagement is low,
but they are not struggling with it in any way – if they do not understand, so what. They do not
value the task.
Interest-Reliant: These readers are comparable to the knowledge reliant readers, but they are
dependent on their own topic interest or situational interest in what the text is addressing at the
moment. They become engaged when their interest is activated, but otherwise find it difficult to
be in control of their own level of effort. So their scores could be high if their interest is piqued
or aligned with the text topic, but otherwise are likely to be low, because they will not put in
much effort to understand and remember what the text is saying. They also might be likely to pay
more attention to their own level of interest, monitoring how interested they are as they go along,
and expecting that the author should be obliged to sustain their interest.
Undergraduates’ Reading Success 54
Table 2. Distribution of Evaluative Comments by Passage
SA Passage
SI Passage
+
% of total passage comments
(raw number of comments)
40.3 (94)
30.3 (66)
-
9.0 (21)
28.9 (63)
+
8.6 (20)
2.7 (6)
-
6.9 (16)
10.1 (22)
+
3.4 (8)
1.8 (4)
-
0 (0)
2.7 (6)
+
3.9 (16.7)
14.2 (31)
-
2.6 (6)
1.4 (3)
+
5.6 (13)
0.5 (1)
-
1.3 (3)
1.0 (2)
Author
1.3 (3)
0.5 (1)
Reading Strategies
3.0 (7)
6.0 (13)
Other
1.3 (3)
0 (0)
Comment
Category
Accessibility
Interest
Familiarity
Text Quality
Argument
Undergraduates’ Reading Success 55
Table 3. Correlations of Initial and Outcome Variables
GPA
NFC
Interest
Domain
SA Topic
SI Topic
SA
SI
Knowledge Knowledge Knowledge Outcome Outcome
GPA
—
NFC
.023
—
Interest
.014
.392*
—
Domain
Knowledge
.234
.078
.094
—
SA Topic
Knowledge
.124
.215
.156
.391*
—
SI Topic
Knowledge
-.003
.103
.207
.369*
.201
—
SA Outcome
.025
.117
.005
.244
.278*
.072
—
SI
Outcome
-.014
-.045
.236
.059
.181
.143
.334*
*p < .05
—