Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading
Comprehension
Topic Sentence:
What is the effect of text format (traditional linear text vs. non-linear hypertext) on
reading comprehension?
Introduction
Emerging technologies are being used more and more in our classrooms. With the
advent of hypermedia and the internet, students are now exposed to written information
in a new way. A proliferation of texts in hypertext format are now being produced.
Traditional print, what we see in traditional textbooks and learning materials, presents
information in a linear manner. The information is offered to the readers in an order
decreed by the writer. Readers typically progress through their reading of traditional
print from beginning to end in sequential order. In contrast, hypertext is nonlinear and
provides more flexibility to the reader in choosing where to go next. The reader
experiences more choices as they navigate through the material, clicking on hyperlinks
(links to other pages of information) based on their needs and interests.
Research comparing linear and hypertext has produced varied results. Studies have
explored avenues such as the impact of readers' prior knowledge, their cognitive abilities,
working memory capacity, and the makeup of the text itself on reading comprehension.
Research Question:
This literature review revolves around the following research question: "What is the
effect of text format (traditional linear text vs. non-linear hypertext), if any, on reading?"
Idealized Independent Variable:
The idealized active independent variable explored in this literature review is text format,
operationally defined as the categorical variables: traditional text (linear) or hypertext
(nonlinear, structured or networked hypertext).
Idealized Dependent Variable:
The idealized dependent variable explored is reading comprehension. The dependent
variable is ambiguous by its very nature and therefore not easily defined in operational
terms. Much research has been done on linear text reading comprehension; user models
have been used to determine what information will be learned from linear text. For the
scope of this literature review, however, participants' reading comprehension will be
limited to and evidenced by their scores on comprehension assessments.
Idealized Directional Hypothesis:
Participants using non-linear structured hypertext will perform better on written reading
comprehension assessments than those using non-linear networked hypertext or those
using traditional linear text.
1
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading
Comprehension
Discussion of Articles
Each of the studies reviewed examined more than one hypothesis in multiple
experimental setups. For the purpose of this review, I will evaluate and analyze only the
data and results which correspond to the idealized independent and dependent variables.
Differences in how the IVs were operationally defined:
All the research studies consistently used traditional-style text format as their linear
category of the independent variable. However, only the Lee and Tedder (2003) and
Calisir and Gurel (2003) studies used hypertext categories similar to that of the idealized
independent variable. Lee and Tedder classified their hypertext format as 'structured' or
'networked' while Calisir and Gurel grouped their hypertext format as 'hierarchical' or
'mixed'.
The studies by Lowry (2004) and McEneaney (2003), are only semi-comparable to the
idealized independent variable. Both of these studies used only 'non-linear' to define their version
of hypertext. No mention was made as to the topology of the hypertext itself. The other two
studies (Wenger, 1996) and (Kasper, 2003) were broader and more specific in their categorization
of hypertext, using up to 4 and 5 categories of hypertext respectively.
Differences in how the DVs were measured:
All of the studies measured reading comprehension by analyzing scores on
comprehension tests. Due to the nature of the experiments, i.e. tailoring their questions to
the specific content of their linear and hypertexts, none of the question sets were
identical. The Wenger study (1996), collected both online performance measures
(reading rate, concurrent task accuracy, and response time and time required to make
sequencing decisions) and reading outcome measures (test for recall, test for
comprehension, and test for memory retention of text structure). However, the
McEneaney study (2003), used two different question sets to measure reading
comprehension instead of one. Another significant difference in the way the dependent
variables were measured is within the question sets themselves. Most of the studies used
short multiple choice recall tests, one study used both multiple choice and fill-in-the
blank items, (Calisir, F. and Gurel, Z., 2003), and one study used short answer question
sets (McEneaney, J., 2003).
Sources of internal validity compromise (e.g., problems with the design and
execution of the study):
All of the studies contain questionable aspects of internal validity. Of primary concern in
all the studies is the reliability of the recall tests. The methodology used to create the
questions sets were remains vague. From the sample question sets provided by the
researchers, it is apparent the majority of the questions used to assess reading
comprehension could all be considered at the lowest level of bloom's taxonomy,
knowledge and recall, and very few questions were at the comprehension level. The
2
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading
Comprehension
'purpose' for reading may also have had an effect on the results. The participants were
'rewarded' with course credit in order to promote their motivation to do their best.
However, reading for pleasure and reading for finding information are very distinct and
may influence memory recall.
In addition, the only study in which the participants were told to study the documents
with the goal of comprehension in mind was the Calisir (2003) study. Other types of
tasks or reading purposes as perceived by the participant may have had an effect on
results.
Another common threat to internal validity among all of the studies (of which only the
McEneaney, (2003) study specifically discussed) may have been the participants'
familiarity with technology tools and practices. The researches failed to mention if they
trained their participants in navigating through hypertext.
A third common concern with respect to internal validity is the amount of time readers
were allowed to spend reading the content. Slower readers and unmotivated readers'
scores may be negatively affected if they are not required to read the entirety of the text.
Only the Wenger (1996) study discussed whether subjects read all of the nodes (ie links'
or hyperlinks' content) within the hypertext and even then, it was mentioned that their
subjects covered on average only 85% of the available nodes.
Furthermore, Lowry, W. (2004) and McEneaney, J. (2003) failed to identify what format
of hypertext they used in their respective studies. Other types of topologies (e.g.
structured, networked, or mixed hypertext) may have had different effects on results.
A final major danger to internal validity was found in the Kasper (2003) study. While
there were different categories of hypertext in their study, the researcher designed each
type of hypertext to be focused on a different subject matter instead of designing different
types of hypertexts focused on the same subject. Some subjects may have been more
difficult for students to comprehend and may have influenced their performance and
reactions to the different types of hypertext used
Differences in samples, populations, and assessments of external validity:
Limitations which could affect the external validity of the results and which should be
taken into account for future studies include the limited size of the sample pool. Here
again, all five studies shared a common thread -- all of the studies were limited by their
populations and sample size; the Lee and Tedder (1993), study severely so. After
dividing their sample size into three conditions (traditional-linear, structured hypertext,
and networked hypertext) and then further subdividing those participants into low,
medium, and high working memory span groups, certain categories had too few a sample
size. Similarly in the McEneary, (2003) study, the small sample size resulted in a
3
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading
Comprehension
statistical power below the recommended level). For the most part, the studies were
carried out with college level students as participants. A more representative sample of
multiple ability reading levels in a more diverse population needs to be done in order to
increase external validity. According to Lowry (2004), younger audiences may focus
more on format and presentation of the material than on the actual content of the text.
Implications of the degree to which the studies support or deny the hypothesis:
Analysis of the studies provided contradictory results. Results of the Calisir and Gurel
(2003) and the Lowry (2004) concluded that readers of the non-linear story were no more
successful than were readers of the linear story at recalling story information. Calisir and
Gurel further concluded that there presented no significant difference between
knowledgeable and non-knowledgeable participants in the hierarchical and the mixed
conditions. Conversely, Lee and Tedder (2003), and McEneany (2003), found that higher
recall scores were achieved by participants reading traditional linear text. Alternatively,
the studies conducted by Kasper (2003), and Wenger (1996) both supported the idealized
hypothesis. Kasper's study indicated that all forms of hypertext produced higher scores
on comprehension tests than did print texts. Wenger's study also agreed that
comprehension was higher for participants using hypertexts than linear texts but with the
added caveat that the comprehension advantage for hypertext was apparent only for the
description texts presented as hypertexts, with the description texts presented as linear
text actually showing lower performance than causation texts presented as hypertext.
Within the topology of different types of hypertexts, results were contradictory as well.
Lee and Tedder concluded no significant different between structured and networked
hypertext when total reading time was taken into consideration, while Kasper concluded
that student feedback and performance data indicated that controlled hypertexts with
glosses led to the highest level of text comprehension.
Final Conclusions
The studies all investigate the effect that hypertext has on the reader's comprehension of
the material, yet the results significantly with each other. Assessing reading
comprehension is difficult matter. Many years of research have been spent on the
evaluation of readers' comprehension, all of it dealing with linear traditional text. The
contradictory results found in the studies reviewed with respect to reading comprehension
as it pertains to nonlinear hypertext only serves to prove how much more research in this
field is needed.
Research needs to be done in order to determine where and in what scenarios hypertext
will be more successful than linear text. Some features of hypertext which will enable it
to succeed over traditional print text include: the ability for readers to find relevant
information in an efficient manner and search through the entire body of the text with
ease, the ability to represent textual information that is not as easily presented in linear
form, and its ability to provide additional information to the reader so that the text
4
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading
Comprehension
automatically contains information that is more appropriate to the reader's level of
reading (Foltz, 1996).
References:
Calisir, F. and Gurel, Z. (2003). Influence of Text Structure and Prior Knowledge of the
Learner on Reading Comprehension, Browsing, and Perceived Control. Computers in
Human Behavior, 19(2), 135-145.
Foltz, P.W. (1996) Comprehension, Coherence and Strategies in Hpyertext and Linear
text. In Rouet, J.-F., Levonen, J.J., Dillon, A.P. & Spiro, R.J. (Eds.) Hypertext and
Cognition. Hillsdale, NJ: Lawrence Erlbaum Associates.
Kasper, L. (2003). Interactive Hypertext and the Development of ESL students' Reading
Skills. The Reading Matrix, 3(3).
Lee, MJ and Tedder, MC. (2003). The Effects of Three Different Computer Texts on
Readers' Recall: Based on Working Memory Capacity. Computers in Human Behavior,
19(6), 767-783.
Lowry, W. (2004). More Control, But Not Clarity in Non-linear Web Stories.
Newspaper Research Journal. 25(2), p83.
McEneaney, J. (2003). Does Hypertext Disadvantage Less Able Readers? Journal of
Educational and Computing Research, 29(1), 1-12.
Wenger, M. (1996). Comprehension and Retention of Non-linear Texts: Considerations
of Working Memory and Material-Appropriate Processing. The American Journal of
Psychology, 109(1), 93-131.
5
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading Comprehension
Author(s)
Lowry, W. (2004). More Control,
But Not Clarity in Non-linear
Web Stories. Newspaper
Research Journal. 25(2), p83.
RQ: What is the relationship, if
any between the linearity of
format of the Web story and the
degree of perceived credibility by
readers?
Kasper, L. (2003). Interactive
Hypertext and the Development of
ESL students' Reading Skills. The
Reading Matrix, 3(3).
RQ: Study design to explore and
asses the possible effects of
hypertext on the development of
ESL students' reading skills.
Independent Variable
The independent variable is a
categorical variable with the two
nominal categories:
Level 1: Linear structured Web
story
Level 2: Non-linear structured
Web story
The independent variable consists
of five hypertext conditions:
Level 1: glosses (links provide pop
up vocabulary)
Level 2: controlled hypertexts
(links lead to predetermined and
limited number of texts on the topic
Level 3: free hypertexts (students
read main text and are directed to
freely explore the internet for other
texts related to the topic
Level 4: controlled hypertexts with
glosses
Level 5: free hypertexts with
glosses
Dependent
Variables
Hypothesis
Tested
Threats to Validity
Hypothesis
Supported?
Subjects' knowledge
acquisition or ability to
recall content from the
story – measured by scores
on question set
Readers of nonlinear Web stories
will demonstrate
significantly
greater knowledge
acquisition than
will readers of
linear Web stories.
The internal validity of the study may have
been affected. The researchers controlled the
amount of time readers were allowed to
spend reading the story instead of allowing
them as much time as they needed to process
and read the information. This would
negatively affect the results of slower
readers.
No. Readers of the nonlinear story were no more
successful than were
readers of the linear story
at recalling story
information.
Student reading
comprehension – measured
by scores on
comprehension tests
What is the role of
hypertext in
developing ESL
students' reading
skills through a
comparison of
students'
performance with
different types of
hypertexts.
And 1 control:
Printed Text
Lee, MJ and Tedder, MC. (2003).
The Effects of Three Different
Computer Texts on Readers'
Recall: Based on Working
Memory Capacity. Computers in
For Hypothesis 1, 2:
The independent variable is a
categorical variable with three
nominal categories:
Level 1: Participants
Assess and
evaluate the
effects of different
types of hypertext
on reading
comprehension
and performance
The dependent variable
being investigated is an
ordinal variable studying:
The number of correct
H1. The
participants
reading from
hypertext
documents will
External validity of the study is compromised
by the following factors:
1. Nature of the sample, which consisted
entirely of undergraduate senior
communications students. The researchers
state that younger audiences may focus more
on format and presentation of the material
than on the actual content of the story.
Internal validity is questionable in this study.
The research designed each type of hypertext
to be focused on a different subject matter
instead of designing different types of
hypertexts focused on the same subject.
Some subjects may have been more difficult
for students to comprehend and may have
influenced their performance and reactions to
the different types of hypertext used.
Overall, the validity and reliability of the data
is questionable. Of primary concern is the
reliability of the recall test. No mention of
how the test was created was mentioned.
Yes. All forms of
hypertext produced
higher scores on
comprehension tests than
did print texts. Student
feedback and
performance data
indicated that controlled
hypertexts with glosses
led to the highest level of
text comprehension.
Overall, this study
indicates that the
structure and presentation
of text influence how well
information is recalled.
6
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading Comprehension
Author(s)
Human Behavior, 19(6), 767-783.
RQ: How does the presentation of
the same information in different
types of text formats, (the
traditional linear text, the
structured hypertext, and the
networked hypertext,) affect a
reader's recall based on his or her
working memory capacity?
Wenger, M. (1996).
Comprehension and Retention of
Non-linear Texts: Considerations
of Working Memory and
Material-Appropriate Processing.
The American Journal of
Psychology, 109(1), 93-131.
RQ: What are the specific process
that support comprehension and
retention of hypertext information
Independent Variable
randomly assigned to traditional
text,
Level 2: Participants
randomly assigned to structured
hypertext, and
Level 3: Participants
randomly assigned to networked
hypertext.
The independent variable is a
categorical variable with the
following nominal categories:
Level 1: Three linear format
causation texts
Level 2: Three hypertext format
causation texts
Level 3: Three linear format
description texts
Level 4: Three hypertext format
Dependent
Variables
answers on the recall test –
measured by a sixteen
question recall multiple
choice factual recall test
Hypothesis
Tested
have lower recall
scores than those
reading identical
information from
traditional lineartext documents.
H2. The
participants
reading from the
structured
hypertext will
have higher recall
scores than those
reading identical
information from
the networked
hypertext.
Collected both online
performance measures
(reading rate, concurrent
task accuracy, and
response time and time
required to make
sequencing decisions and
Reading outcome measures
(recall, comprehension,
and memory for text
structure)
Hypertext will
impose more
demands for
relational
processing than
would linear text.
Relational vs.
item-specific
processing
Threats to Validity
Hypothesis
Supported?
This study had several limitations which
could affect the external validity of the
results and which should be taken into
account for future studies.
1. The sample size. After dividing the
sample size into three conditions (traditionallinear, structured hypertext, and networked
hypertext) and then further subdividing those
participants into low, medium, and high
working memory span groups, certain
categories had too few a sample size.
2. The study only examined simple
memory factual recall. Learning/memory is
much more complex. The effects of
hypertext may be different for higher
cognitive thinking, i.e., synthesis or problem
solving tasks.
3. The treatment of the information is not
necessarily how one would find it in a real
world setting. All hypertext links were
limited and focused within the same
document. Such is not true for real world
Web pages.
4. The 'purpose' for reading may also have
had an effect on the results. The students
were 'rewarded' with credit in order to
promote participant motivation to do their
best. However, reading for pleasure and
reading for finding information are very
distinct and may influence memory recall.
Internal validity is questionable in this study.
No guarantee or evidence was provided that
subjects read all of the nodes within the
hypertext. It was only mentioned that the
subjects covered on average 85% of the
available nodes)
Text format had a
significant effect on recall
scores.
External validity may be compromised by the
selection of participants. All subjects
participated in the study in order to complete
course requirements. It is not said if the
Analysis of the data
shows hypothesis 1 is
supported. Higher recall
scores were achieved by
participants reading
traditional linear text over
those participants who
read the structured
hypertext.
Analysis of the data
shows hypothesis 2 is not
supported. Higher recall
scores were achieved by
participants reading
networked hypertext over
those participants who
read structured hypertext.
However, this difference
disappeared when total
reading time was
considered.
Comprehension for
hypertext was higher
when reading with the
spatial load rather than
the digital load, whereas
the opposite was true for
linear texts
Comprehension was
higher for the description
texts than for the
7
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading Comprehension
Author(s)
and the manner in which those
processes different from those
involved in the comprehension
and retention of traditional linear
text?
Independent Variable
Dependent
Variables
Hypothesis
Tested
description texts
Threats to Validity
selection was random or if students
volunteered to be subjects.
Test for recall
Test for comprehension
RQ: What is the relationship
between print and hypertext
reading skills in university
students?
Calisir, F. and Gurel, Z. (2003).
Influence of Text Structure and
The independent variable is a
categorical variable with the two
nominal categories:
Level 1: Printed version of the
handbook
Level 2: Hypertext version of the
handbook
The independent variable is a
categorical variable with the three
Reading comprehension as
measured by results to two
sets of academic advising
questions
Reading comprehension
(as measured by score by
causation texts
Results indicate that
comprehension was
higher for hypertexts than
for linear texts. However,
comprehension advantage
for hypertext was
apparent only for the
description texts present
as hypertexts, with the
description texts
presented as linear text
actually showing lower
performance than
causation texts presented
as hypertext
Test for retention of text
structure
McEneaney, J. (2003). Does
Hypertext Disadvantage Less Able
Readers? Journal of Educational
and Computing Research, 29(1),
1-12.
Hypothesis
Supported?
Given the same
reading material,
do 'less able'
readers make more
effective use of
hypertext or
traditional print
formats in a
reading and
questionanswering task?
The internal validity of the study may have
been affected by participants' familiarity with
technology tools and practices. No mention
of participants' level of comfort with using
the technology was mentioned. The
researches failed to 'train' their participants in
their use.
How does text
structure and prior
Only tested two types of hypertext topologies
(hierarchical and mixed). Other types of
'Less able' readers found
the hypertext version of
the advising handbook
more difficult to use than
the print version.
Limitations which could affect the external
validity of the results and which should be
taken into account for future studies include
the limited size of the sample pool. (The
small sample size resulted in a statistical
power below the recommended level) and the
question set used to assess reading
comprehension (all were at the lowest level
of bloom's taxonomy, knowledge, and very
few questions were at the comprehension
level)
In linear text,
knowledgeable
8
Connie Gray
PMCH 6481: Assignment 3
Nov. 12, 2004
A Literature Review of the Effect of Text Format on Readers' Reading Comprehension
Author(s)
Prior Knowledge of the Learner
on Reading Comprehension,
Browsing, and Perceived Control.
Computers in Human Behavior,
19(2), 135-145.
RQ: What is the influence of text
structure and prior knowledge of
the learner on reading
comprehension, browsing, and
perceived control?
Independent Variable
nominal categories:
Level 1: Tradition Linear Text
Level 2: Hierarchical hypertext
Level 3: Mixed Hypertext
Dependent
Variables
Hypothesis
Tested
Threats to Validity
16 multiple choice and 22
fill-in-the-blank items
based on content on all the
nodes in the hypertext
readings)
knowledge of the
learner affect
reading
comprehension,
browsing, and
perceived control?
topologies (e.g. nonlinear grid and generative
topologies) may result in different effects on
reading comprehension, browsing, and
perceived control.
Number of different nodes
opened during reading
Perceived control as
measured by questionnaire
Participants were told to study the documents
with the goal of comprehension in mind.
Other types of tasks or reading purposes may
have an effect on results.
Population consisted of only graduate
students at Istanbul Technical University. A
more representative sample of multiple
ability reading levels in a more diverse
population needs to be done in order to
increase external validity.
Hypothesis
Supported?
participants '
performance was better
than that of nonknowledgeable
participants.
There was no significant
difference between
knowledgeable and nonknowledgeable
participants in the
hierarchical and the
mixed conditions.
9