Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
INTRODUCTION
Research Question:
What is the effect of text format (traditional linear text vs. non-linear hypertext), if any,
on the reading comprehension of high school Limited English Proficient (LEP) students?
Abstract
Emerging technologies are being used more and more in our classrooms. With the
advent of hypermedia and the internet, students are now exposed to written information
in a new way in the form of hypertext. Hypertext differs from traditional print media in
that it is a nonlinear format for text which provides more flexibility to the reader in
choosing where to go next. Research comparing linear traditional text and non-linear
hypertext has produced varied and at times contradictory result. Limited to no research
exists on how non-linear hypertext may vary across demographic factors such as age and
level of education. The study outlined in this proposal will help determine what is the
effect of text format (traditional linear text vs. non-linear hypertext), if any, on the
reading comprehension of high school Limited English Proficient (LEP) students – a
demographic area where there is no published research. It is expected that high school
LEP students using non-linear hypertext formatted texts will perform better on written
reading comprehension assessments than those using traditional linear texts.
1
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
Literature Review
Section I – Background and Need for Study:
Emerging technologies are being used more and more in our classrooms. With the
advent of hypermedia and the internet, students are now exposed to written information
in a new way in the form of hypertext. Hypertext differs from traditional print media in
that it is a nonlinear format for text which provides more flexibility to the reader in
choosing where to go next.
Research comparing linear traditional text and non-linear hypertext has produced varied
and at times contradictory results. The studies reviewed here all investigate the effect
that hypertext has on the reader's comprehension of the material. Results of the Calisir
and Gurel (2003) and the Lowry (2004) studies concluded that readers of the non-linear
story were no more successful than were readers of the linear story at recalling story
information. Conversely, Lee and Tedder (2003), and McEneany (2003), found that
higher recall scores were achieved by participants reading traditional linear text.
Alternatively, the studies conducted by Kasper (2003), and Wenger (1996) both
supported the idealized hypothesis – that comprehension was higher for participants
using hypertexts rather than linear texts.
The contradictory results found in the studies reviewed with respect to reading
comprehension as it pertains to nonlinear hypertext only serves to prove how much more
research in this field is needed. Assessing reading comprehension is difficult matter.
Many years of research have been spent on the evaluation of readers' comprehension, all
of it dealing with linear traditional text.
Research needs to be done in order to determine where and in what scenarios hypertext
will be more successful than linear text. Some features of hypertext which will enable it
to succeed over traditional print text include: the ability for readers to find relevant
information in an efficient manner and search through the entire body of the text with
ease, the ability to represent textual information that is not as easily presented in linear
form, and its ability to provide additional information to the reader so that the text
automatically contains information that is more appropriate to the reader's level of
reading (Foltz, 1996).
Studies have explored avenues such as the impact of readers' prior knowledge, their
cognitive abilities, working memory capacity, and the makeup of the text itself on
reading comprehension of hypertext (Calisir and Gurel, 2003, Lee and Tedder, 2003, and
Kasper, 2003). However, limited to no research exists on how non-linear hypertext may
vary across demographic factors such as age and level of education. The study outlined
in this proposal will help determine what is the effect of text format (traditional linear
text vs. non-linear hypertext), if any, on the reading comprehension of high school
2
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
Limited English Proficient (LEP) students – a demographic area where there is no
published research.
Section II – Methods:
An inherent problem with the differences in results from the studies reviewed lies in the
underlying goals for each study. Their purpose for researching hypertext effects on
reading comprehension was slightly different in each study and it shows in their results.
All the research studies reviewed consistently used traditional-style text format as their
linear category of the independent variable. However, only the Lee and Tedder (2003)
and Calisir and Gurel (2003) studies used similar hypertext categories. Lee and Tedder
classified their hypertext format as 'structured' or 'networked' while Calisir and Gurel
grouped their hypertext format as 'hierarchical' or 'mixed.' However, the studies by
Lowry (2004) and McEneaney (2003), used only 'non-linear' to define the topology of
their hypertext and consequently determinations as to their comparisons to either
structured or networked hypertext can not be made. In contrast, other two studies
(Wenger, 1996) and (Kasper, 2003) were broader and more specific in their
categorization of hypertext, using up to 4 and 5 categories of hypertext respectively.
Within the topology of different types of hypertexts, results were incongruous as well.
Lee and Tedder concluded no significant different between structured and networked
hypertext when total reading time was taken into consideration, while Kasper concluded
that student feedback and performance data indicated that controlled hypertexts with
glosses (pop ups with definitions of unfamiliar terms) led to the highest level of text
comprehension.
Another significant difference between the studies is in the way the dependent variable of
reading comprehension is measured through the analysis of scores on reading
comprehension tests which brings into question the studies' internal validity. Most of the
studies used short multiple choice recall tests, one study used both multiple choice and
fill-in-the blank items, (Calisir, F. and Gurel, Z., 2003), and one study used short answer
question sets (McEneaney, J., 2003). Moreover, from the sample question sets provided
by the researchers, it is apparent the majority of the questions used to assess reading
comprehension could all be considered at the lowest level of bloom's taxonomy,
knowledge and recall, and very few questions were at the comprehension level.
This study is designed to explore and assess the possible effects of hypertext on the
development of high school limited English proficient (LEP) students' reading
comprehension. For the scope of this research study, a significant difference in
methodology from the previous research of hypertext's influence in reading
comprehension will be in the reading materials and the question sets developed for pre
and post assessment of reading comprehension. The written assessments will measure
how well the students comprehended the material they read through higher order criterion
referenced question sets developed by reading specialists working in collaboration with
3
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
the language instructor. All reading materials and question sets will be standardized to a
9th reading level using the Klesch-Kincaid Reading Level built into the newer versions of
Microsoft® Word and easy-to-use readability graphs for educators created by Edward
Fry and available through McGraw-Hill.
Variables and Hypothesis:
Dependent Variable:
The dependent variable being investigated is an ordinal variable studying:
LEP Students' reading comprehension as measured through the number of correct
answers on written reading comprehension assessments.
Independent Variables:
The independent variable is a categorical variable with the following nominal categories:
Group 1 students will be given texts in traditional linear format.
Group 2 students will be given texts in hypertext nonlinear format.
Directional Hypothesis:
High school Limited English Proficient (LEP) students using non-linear hypertext
formatted texts will perform better on written reading comprehension assessments than
those using traditional linear texts.
METHODS
Population and Sample:
Participants in the study will be composed of high school Limited English Proficient
(LEP) students at a 3A South Texas high school. The sample will be drawn from LEP
students at the 'beginning' level of English proficiency as determined by their scores on
the Reading Proficiency Tests in English (RPTE) and have had two years or less of
formal English instruction. The results will only generalize to other LEP students and
other learners where English is a second and not primary language. They may not apply
to proficient readers of English or learners whose primary language is English. Students
will be randomly assigned to the two text format groups based on their class schedule.
Morning classes will receive linear text materials and afternoon classes will receive
hypertext materials. The number of computers and/or laptops available will limit the
hypertext group, therefore a maximum of 24 students per class will be allowed for a total
of 96 students participating in the study.
Research Design:
The experiment is based on a differences research question consisting of a pre to post
covariance design as represented in Design Five of The Whole Art of Deduction. In the
experiment, each student will be randomly assigned to either the linear traditional text
4
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
group or the nonlinear hypertext group. Students in both groups will be given a preassessment to account for prior knowledge. They will then be given their assigned
readings in the designated text format. After the students have finished reading, they will
be asked to take a written assessment.
The written assessment will measure how well the students comprehended the material
they read by asking them to recall facts and make inferences on the text they read. Each
written assessment will be developed by an experienced reading specialist working in
collaboration with the language instructor. The written assessments will be at a 9th grade
reading level and consist of multiple choice, fill-in-the-blank, and short answer questions.
See Appendix A for a few sample questions.
The steps will be repeated for a total of six independent units. ie) Each group will
undergo the pre-test, reading (nonlinear hypertext or linear traditional text), post-test for
six different topic units.
Diagram of Research Design
RA
Event
1
2
3
Group 1
O1
X1
O2
Group 2
O1
X2
O2
Efforts will be made to preserve the validity and reliability of the data. Of primary
concern is the reliability of the recall assessment. Each question on the recall test will be
evaluated independently of the study to maintain integrity of the testing procedures.
The following controls will be in place for the duration of the study:
1. The only difference between the texts read by the students will be text format
(hypertext or linear text). All content (information, graphics, etc.) will be kept the
same for both groups.
2. To ensure consistency in the population, students will be classified as LEP
students (beginning, intermediate, or advanced) based on their Reading
Proficiency Tests in English (RPTE) scores.
3. All participants will have the same instructor, be exposed to the same teaching
strategies, and be given the same instructions and test procedures.
4. The reading level of each text unit to be read by the students will be measured and
be revised as necessary so that a 9th grade reading level of rigor is met.
Extraneous variables considered will include:
1. Participant familiarity with using hypertext and/or the computer to read it. To
prevent a skew of the results due to level of technology proficiency, each student
5
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
assigned to the hypertext group will be given instruction in how to use and
navigate their way through hypertext before the study commences.
2. The total reading time spent on reading the material. All efforts will be made to
ensure reading time is not limited and that each student will be given ample time
with the texts.
Data Analysis:
Means and standard deviations will be calculated for the proportion of comprehension
questions answered correctly as a function of text format. These data will be presented in
a table (See Table 1 for example). ANOVA and an Independent-Samples t-test will be
used to test whether there is a significant difference among the means.
TABLE 1. Proportion of Comprehension Questions Answered Correctly
as a Function of Text Format (Unit 1)
Group
Mean
Standard
Deviation
N
Linear Traditional Text Format (Class
#1)
Linear Traditional Text Format (Class
#2)
Non-linear Hypertext Format (Class #3)
Non-linear Hypertext Format (Class #4)
(Note: Table will be repeated with appropriate data and data analysis for each unit)
References:
Calisir, F. and Gurel, Z. (2003). Influence of Text Structure and Prior Knowledge of the
Learner on Reading Comprehension, Browsing, and Perceived Control. Computers in
Human Behavior, 19(2), 135-145.
Foltz, P.W. (1996) Comprehension, Coherence and Strategies in Hpyertext and Linear
text. In Rouet, J.-F., Levonen, J.J., Dillon, A.P. & Spiro, R.J. (Eds.) Hypertext and
Cognition. Hillsdale, NJ: Lawrence Erlbaum Associates.
6
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
Fry, E. (1977) Elementary Reading Instruction. NY : McGraw Hill, p.217
Kasper, L. (2003). Interactive Hypertext and the Development of ESL students' Reading
Skills. The Reading Matrix, 3(3).
Lee, MJ and Tedder, MC. (2003). The Effects of Three Different Computer Texts on
Readers' Recall: Based on Working Memory Capacity. Computers in Human Behavior,
19(6), 767-783.
Lowry, W. (2004). More Control, But Not Clarity in Non-linear Web Stories.
Newspaper Research Journal. 25(2), p83.
Marion, R. (2004).The Whole Art of Deduction: Research Skills for New Scientists.
{Online Text}
http://www.sahs.utmb.edu/pellinore/intro_to_research/wad/wad_home.htm
McEneaney, J. (2003). Does Hypertext Disadvantage Less Able Readers? Journal of
Educational and Computing Research, 29(1), 1-12.
Wenger, M. (1996). Comprehension and Retention of Non-linear Texts: Considerations
of Working Memory and Material-Appropriate Processing. The American Journal of
Psychology, 109(1), 93-131.
7
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
Appendix A
Sample Question 1: In paragraph 14, the author describes Vin as “wringing his hands” to
show that Vin is —
A) nervous about the mandarin’s attitude
B) disgusted with his poor brother
C) concerned about his friend’s response
D) irritated with his wife
Sample Question 2: Paragraphs 16 through 21 are mostly about —
A) the many loyal friends Vin has made
B) Vin’s attempts to further cheat his brother
C) Vin’s rise to popularity and riches
D) the unwillingness of Vin’s friends to help him
8
Connie Gray
PMCH 6481: Assignment 4
Dec. 10, 2004
Research Proposal
Effect of Text Format on Reading Comprehension of LEP Students
Appendix B (Forms)
Note:
1) this research study involves minimal or no risk for participants and is conducted in an
educational setting that focuses on instructional strategy or effectiveness of educational
techniques or curricula and therefore may qualify for a 'Request for Exempt Status'
review.
2) As per school district regulations, a full explanation of the study, its purpose and
procedures will be provided to all students and their parents along with consent forms.
9