ON-LINE WRITING CENTER RESPONSES AND ADVANCED
EFL STUDENTS' WRITING: AN ANALYSIS OF COMMENTS,
STUDENTS' ATTITUDES, AND TEXTUAL REVISIONS
by
LILIANA BEATRIZ ANGLADA, B.A., M.A.
A DISSERTATION
IN
ENGLISH
Submitted to the Graduate Faculty
of Texas Tech University in
Partial Fulfillment of
the Requirements for
the Degree of
DOCTOR OF PHILOSOPHY
Approved
May, 1999
Copyright 1999, Liliana Anglada
ACKNOWLEDGMENTS
This dissertation could not have been completed without the invaluable help and
support I received from a number of people and organizations throughout my doctoral
program at Texas Tech University. Furthermore, the assistance and cooperation of
fiiends, family, colleagues, and students during the data collection process in Argentina
constitute two essential elements on which this research was built.
First of all, I thank Dr. Patricia Goubil-Gambrell, my dissertation committee
chair, for her assistance and feedback. Her sohd knowledge of research methodology
assisted me the most during the data collection process, the design of surveys and
interviews, and the first stages of my writing. Dr. Mary Jane Hurst, my second
committee member, was one of my first instructors at Texas Tech University, and her
unconditional moral support, academic assistance, and professionalism have been steady
since then. For this particular work, she did an impeccable job guiding me through the
difficult process of writing, in the absence of my dissertation committee chair. There are
no words to express how indebted I feel towards her and her professional attitude towards
my work. Dr. Fred O. Kemp, my third committee member, always showed his support
and willingness to discuss possible ways of approaching various issues in my
dissertation. He always had words of encouragement about my work and showed faith in
my project.
Two extremely important groups of people deserve mention at this point. I am
greatly indebted to the Texas Tech University Writing Center as an institution, to Dr.
Lady Falls Brown, as its general director, to Lynnea Chapman-King, as assistant director
of the center and director of the On-line Writing Center, and to all the consultants who
ii
participated in this project. Their willingness to conduct fiiture projects similar to the one
I described in this work shows their commitment to the progress and development of
good writers. The teacher-training college where this study was conducted, Instituto Juan
Zorrilla de San Martin, not only offered the research site for my study but also supported
me economically since I worked as an instructor for the school while the study was
underway. Without the help of the school director, Lie. Juana A. Evangelisti, its faculty
members, administrative staff, and especially without the participation of its students, this
innovative project could never have been accomplished.
I also would like to express my infinite gratitude to four organizations that
supported me financially with grants and scholarships during my studies in the US: The
Instimte of International Education (with a Fulbright Scholarship), the PEO Intemational
Peace Scholarship Fund, the Texas Tech University Graduate School (with a Summer
Research Award), and the Helen Hodges Educafional Charitable Trust.
Three independent raters whose help became invaluable during the difficult task
of coding and analysis of the data were Sarojini Arani, Alexandra Dzenowagis, and Jeff
WilHams. Their time, effort, and willingness to help me in the sometimes tedious and
exacting tasks of classifying text passages were essential. Their work provided me with
the necessary tools to be able to analyze and report tendencies in the data.
The Department of Psychology at Texas Tech University also deserves a special
mention. Dr. Gregory H. Mumma and Jason Smith, a teaching assistant, offered me
invaluable information and guidance concerning some complex statistical analyses.
Dr. Wendell M. Aycock deserves much more thankfulness than words can
capture. While teaching a literature course in Argentina, Dr. Aycock encouraged his
111
students to come to the US forfiorthereducation. I am grateful that I was one of the
students who, following his advice, decided to pursue a graduate degree in an American
university. Dr. Aycock's fiiendship, support, and academic advice have been and still are
invaluable. Not only Wendell but also Jessie, Diane and Daniel (his mother, wife and
son, respectively) have become like members of my family in the affection, hospitality,
and generosity they have always demonstrated towards me.
I would like to acknowledge and give special thanks to my fiance, Jeff Williams,
for his unbreakable spirit that helped me to go through some very difficult times, when
discouragement and lack of progress seemed to set in. I also owe a word of thanks to my
partner, colleague, and fiiend, Sandra Gastaldi, who generously shared the premises of
our institute so that communication via e-mail could be established with the Texas Tech
University On-line Writing Center.
Although placed at the end in the list of acknowledgements, my close and
extended family and fiiends should receive the same gratitude I expressed to all the
people above. Their support at a distance, through letters, telephone calls, and e-mail
messages helped me keep myself together. I greatly thank my loving mother, Santina,
who patiently spent these years of her life waiting for my return, but who never tried to
interfere with my objectives in life. I also thank my sister, Monica, whose letters always
brought me joy, memories, and strength to carry on. Finally, I dedicate this work to the
memory of my father, Delfin Anglada, who—with his indomitable spirit—taught me first
and foremost that everything is possible because, as he used to tell me in Spanish,
''querer es poder' ("where there is a will there is a way").
IV
TABLE OF CONTENTS
ACKNOWLEDGMENTS
ii
ABSTRACT
ix
LIST OF TABLES
xi
CHAPTER
1. INTRODUCTION
1
1.1 Background Information: EFL Populations' Special
Needs, SLA Theories and Assumptions, and
the Role of Technology
4
1.2 Current Approaches to ESL/EFL Writing Instruction that
Justify the Implementation of this Project
16
1.3 Literature Review
24
1.3.1 ESL/EFL Writing
25
1.3.2 Computer-mediated Communication in NES and
ESL/EFL Settings
34
1.3.2.1 CMC in NES Settings
35
1.3.2.2 CMC in ESL/EFL Environments
37
2. METHODOLOGY
44
2.1 Setting and Participants
47
2.1.1 Research Site
47
2.1.2 Selection of Subjects
50
2.2 Implementation: Students' Tasks and Computer Classroom
55
2.3 Data Collection Procedures
60
2.3.1 Participants' Short Essays
60
2.3.2 On-line Writing Center Responses
63
V
2.3.3 Field Notes
66
2.3.4 Surveys
70
2.3.5 Interviews
73
2.4. Data Analysis: Classification and Coding of Data
77
2.4.1 Taxonomies for data analysis
78
2.4.2 Operational Definition of Comment
83
2.4.3 Training Sessions
86
3. RESULTS: EFL STUDENTS' ATTITUDES AND PERCEPTIONS
AS REVEALED IN THE SURVEYS, INTERVIEWS AND
FIELD NOTES
92
3.1 Students' Perceptions and Experience with Revision
Practices (First Survey)
94
3.2 Students' Views on Receiving Advice and Suggestions from
the Texas Tech University On-line Writing Center
(Second Survey)
109
4. RESULTS: TEXAS TECH UNIVERSITY ON-LINE WRITING
CENTER CONSULTANTS' COMMENTS AND EFL
STUDENTS' TEXTUAL REVISIONS
125
4.1 Inter-rater Reliability Measures
128
4.2 Types of Comments Classified according to Pragmatic Intent
(Taxonomy A) and Students' Textual Revisions
132
4.3 Types of Comments Classified according to Content or
Substance (Taxonomy B) and Students'
Textual Revisions
158
5. CONCLUSIONS
191
5.1 Inifial Aims and Expectations of the Study and the Extent
to Which They Were Fulfilled
VI
192
5.2 Main Tendencies in the Students' Perceptions and
Behaviors and Main Characteristics of On-line
Writing Center Consultants' Comments
197
5.3 Limitations of this Study and Caveats Relevant for
Future Research
206
BIBLIOGRAPHY
222
APPENDIX
A: INSTITUTO JUAN ZORRILLA DE SAN MARTIN'S
STUDY PROGRAM FOR TEACHING ENGLISH AS A
FOREIGN LANGUAGE CERTIFICATION
246
B: TEST TOPICS FOR LITERATURE m, AT INSTITUTO
JUAN ZORRILLA DE SAN MARTIN, FOR THE
1997 ACADEMIC YEAR
248
C: ANALYTIC MODEL FOR THE CLASSIFICATION
OF TEACHER COMMENTARY
251
D: TAXONOMY FOR ON-LINE WRITING CENTER
CONSULTANTS'COMMENTARY
253
E: RATING SCALES FOR TEXTUAL REVISIONS
255
F: TABLES WITH THE INITIAL AND FINAL NUMBER OF
COMMENTS PER ON-LINE WRITING CENTER RESPONSE AS
COUNTED BY BOTH THE RESEARCHER AND
AN INTEPENDENT READER
258
G: EXAMPLES OF INITLVL DIFFERENCES BETWEEN
THE RESEARCHER AND AN INDEPENDENT READER'S
SEPARATION OF COMMENT UNITS AND THEIR FINAL
AGREED UPON DECISIONS
261
H: GRIDS FOR THE CLASSIFICATION OF WRITING
CONSULTANTS' COMMENTS AND STUDENTS'
TEXTUAL REVISIONS
267
I: TAXONOMIES CREATED FOR THE PURPOSES
OF THIS STUDY
270
Vll
J: EXCERPTS FROM FIELD NOTES
279
K: STUDENT CONSENT FORM
287
L: FIRST SURVEY
289
M: SECOND SURVEY
296
N: INTERVIEW QUESTIONS
301
O: CLASSIFICATION OF COMMENTS ACCORDING TO
PRAGMATIC INTENT AND STUDENTS' TEXTUAL
REVISIONS FOR THE FIRST EXCHANGE
303
P: CLASSIFICATION OF COMMENTS ACCORDING TO
SUBSTANCE AND STUDENTS' TEXTUAL REVISIONS
FOR THE FIRST EXCHANGE
314
Vlll
ABSTRACT
This dissertation analyzes the suggestions for revision sent by on-line writing
center consultants in the United States to advanced EFL students in Argentina and
examines the students' reactions to this type of feedback. Previous ESL/EFL writing
process research, specifically in the area of revision, has explored issues such as peer
critique and teacher feedback. Quite a few studies have focused on learners' attitudes to
feedback, while others have paid particular attention to feedback incorporation during
revision work. Most of these studies, however, have been conducted in regular classes
where either ESL/EFL instructors or peers responded to drafts. Results from these
studies tend to be inconclusive and cannot be applied to specific monolingual settings.
Furthermore, very few studies have investigated how having a real audience of native
speakers of Enghsh, and receiving suggestions from them, may affect ESL/EFL writing.
The research conducted for this project was an attempt to explore this issue.
This study follows a case study methodology. During the 1997 academic year,
one group of advanced EFL students taking a literature course at the teacher-training
college "Juan Zorrilla de San Martin" (Cordoba, Argentina) e-mailed their short essays to
and received feedback on two occasions from the writing consultants at the Texas Tech
University On-line Writing Center. The participants' attitudes toward these electronic
exchanges were analyzed through survey answers and interviews. The types of
comments from the On-line Writing Center consultants and the textual changes made by
the students were coded and subsequently examined employing three different
taxonomies created for the purposes of this study.
ix
Results show that, despite a few difficulties with the technical implementation of
the project, these EFL students benefited from interaction with native-speaker consultants
via e-mail exchanges. These students not only appreciated the feedback received but also
employed a high percentage of comments to their advantage by making changes that
enhanced the quality of their texts. Although a high percentage of the revisions involved
formal or structural problems—as opposed to global or macrostructural concerns—the
number of modifications the students incorporated in their final drafts supports the use of
on-line writing center responses during the revision stage in EFL settings.
LIST OF TABLES
2.1 Dates the students performed the various tasks during the school year
61
2.2 Time elapsed between the different activities for each exchange
62
3.1 Comparison of numbers and percentages of changes incorporated
by the nine students according to the types of comments that
prompted those changes
112
4.1 Values for the overall Cohen's Kappa coefficient, probability level,
and confidence interval for the different rating tasks
for the six groups of data
131
4.2 Numbers and percentages for cases of complete and partial agreement
and for cases of complete disagreement (first exchange)
134
4.3 Numbers and percentages for cases of complete and partial agreement
and for cases of complete disagreement (second exchange)
135
4.4 Numbers and percentages for the various types of comments classified
according to Taxonomy A (pragmatic intent of comments), for the
first exchange, out of the sum total of cases of partial and
complete agreement combined
138
4.5 Numbers and percentages for the various types of comments
classified according to Taxonomy A (pragmatic intent of comments),
for the first exchange, out of the sum total of cases of complete
agreement on both Taxonomies A and C (i.e., 66)
139
4.6 Number and percentages of positive comments versus other categories
of comments classified according to pragmatic intent,
for the first exchange
140
4.7 Percentages of positive comments for the first and second exchanges
out of the sum total of cases of partial and complete agreement
and out of the sum total of cases of complete
agreement (Taxonomies A and C)
142
4.8 Numbers and percentages for the various types of comments classified
according to Taxonomy A (pragmatic intent of comments), for the
second exchange, out of the sum total of cases of partial
and complete agreement combined (i.e., 98)
144
XI
4.9 Numbers and percentages for the various types of comments classified
according to Taxonomy A (pragmatic intent of comments), for the
second exchange, out of the sum total of cases of
complete agreement (i.e., 71)
145
4.10 Numbers and percentages for types of changes ("effects") in students'
drafts for the first exchange in relation to the classification of
comments according to pragmatic intent (Taxonomy A)
147
4.11 Numbers and percentages for types of changes ("effects") in students'
drafts for the first exchange, out of the sum total of cases of complete
agreement for both Taxonomy A and Taxonomy B
147
4.12 Numbers and percentages for types of changes ("effects") in students'
drafts for the second exchange in relation to the classification
of comments according to pragmatic intent (Taxonomy A)
150
4.13 Number and percentage for types of changes in students' drafts for
the second exchange, out of the sum total of cases of complete
agreement for both Taxonomy A and Taxonomy C
150
4.14 Frequency counts and proportions for the six types of comments
classified according to pragmatic intent resulting in positive
effects in the students' drafts for the first exchange
152
4.15 Frequency counts and proportions for the six types of comments
classified according to pragmatic intent resulting in positive
effects in the students' drafts for the second exchange
154
4.16 Numbers and percentages for cases of complete and partial agreement
and for cases of complete disagreement for the first exchange
160
4.17 Numbers and percentages for cases of complete and partial agreement
and for cases of complete disagreement for the second exchange
160
4.18 Numbers and percentages for types of changes ("effects") for the
first exchange in relation to the classification of comments
according to their substance (Taxonomy B)
164
4.19 Numbers and percentages for types of changes in students' drafts
for the first exchange, out of the sum total of cases of complete
agreement for both Taxonomy B and Taxonomy C
164
4.20 Numbers and percentages for types of comments classified according
to substance out of the sum total of cases of complete agreement
for taxonomies B and C
166
xii
4.21 Number of comments classified according to substance in combination
with number of changes ("effects") for the first exchange, out of the
sum total of cases of complete agreement
167
4.22 Numbers and percentages for collapsed categories (derived from
comments classified according to their substance) for the first
exchange, out of the total nimiber of cases of complete
agreement on Taxonomies B and C (i.e., 70)
168
4.23 Numbers and percentages of global and formal comments out of
the sum total of cases of complete and partial agreement
combined, for the first exchange
170
4.24 Numbers and percentages for types of effects or changes (Taxonomy C)
for the second exchange in relation to the classification of comments
according to substance (Taxonomy B)
175
4.25 Number and percentage for types of changes in the students' drafts
for the first exchange, out of the sum total of cases of complete
agreement for both Taxonomy B and Taxonomy C
176
4.26 Number of comments classified according to their substance in
combination with number of changes ("effects") for the second
exchange, out of the sum total of cases of complete agreement
179
4.27 Numbers and percentages for collapsed categories (derived from
comments classified according to substance) for the second
exchange, out of the total number of complete agreements
on Taxonomies B and C (i.e., 75)
179
4.28 Numbers and percentages of global and formal comments out of the sum
total of cases of complete agreement (for Taxonomies B and C) and
of discrepancy (for Taxonomy C), for the second exchange
181
F. 1 Number of Comments per On-line Writing Center Response
Coimted by Both the Researcher and an Independent Reader
259
F.2 Final Number of Comments per On-line Writing Center Response
agreed upon by both the researcher and an independent reader
260
0.1 Classification of comments according to Taxonomy A (pragmatic
intent) and classification of textual changes according to
Taxonomy C for the first exchange
304
XIU
P.l Classification of comments according to Taxonomy B (substance)
and classification of textual changes according to
Taxonomy C for the first exchange
XIV
315
CHAPTER 1
INTRODUCTION
This dissertation contains an analysis of the types of comments from Native
English speaker (NES) reviewers and the textual changes in the drafts of students in an
EFL^ school in Cordoba, Argentina. After these EFL students and the On-line Writing
Center consultants at Texas Tech University, in the US, had established computermediated communication (CMC) via e-mail, both the students' written work and the
consultants' electronic messages were collected and coded for analysis. As part of this
study, the students' perceptions and reactions to this new writing experience involving
CMC with instructors overseas were also examined.
The general problem this case study addresses revolves around the lack of
practice in and exposure to the target language in EFL environments in monolingual
countries. More specifically, this research study focuses on the possibiHties offered by a
combination of resources (i.e., CMC and NES writing consultants) in strengthening EFL
learners' writing skills. At the same time, this study suggests possible ways through
which the main objectives and tenets of the Process Approach in the teaching of writing
can be better and more efficiently achieved. This project was inspired and prompted by
the realization that the recent contributions of technology to the field of education, in
' EFL, ESL, SLA, NES, NNES, LI, L2, ESOL, and CMC, are commonly used acronyms
that will be employed throughout this paper. EFL stands for English as a foreign language; ESL,
for English as a second language; SLA, for second language acquisition; NES, for native English
speaker; NNES, for non-native speaker; LI, for language one or native language; L2, language
two or second language; ESOL, Enghsh speaker(s) of other languages; and CMC,
computer-mediated communication.
general, and to the field of language learning, in particular, offer an innumerable array of
possibilities that should be exploited in the language classroom. In other words, the
untapped pedagogical advantages inherent in the use of computers as a means of
communication inspired this project. Three main research questions motivated and
guided this study: What types of feedback do on-line writing center^ consuUants often
provide EFL learners?; what types of feedback do EFL learners more often incorporate in
their texts?; and how receptive are EFL learners to revision practices prompted from
these electronic exchanges? The answers to these questions are examined in Chapters 3
and 4 of this work, through the analysis of the data obtained during the study. Chapter 2
addresses methodological issues pertaining to the type of research conducted for this
project.
The present chapter includes three main sections. The first section provides
background information necessary to understand the focus and scope of this study and
specifically addresses the following issues and problems: special needs of L2 learners,
particularly EFL populations in monolingual countries; technological advances that may
" The phrase "on-line writing center" has been used throughout this document to refer to
the group of tutors or consultants who work for a writing center but conduct their tutonals
electronically, via e-mail, instead of actually meeting the students or customers face-to face in a
physical place. Although the acronym "OWLs," which stands for "On-line Writing Labs" is
widely used in the literature, I have opted to use the longer phrase "on-line writing centers" rather
than "OWLs" because the former seems to capture more faithfully the kind of work that writing
consultants do for their customers. In an e-mail message explaining the origin of the acronym
"OWLs," Lady Falls Brown, director of the Texas Tech University Writing Center stated, "The
acronym OWL is recognized in the writing center community, and it originated with Muriel
Harris at Purdue. She uses the term Writing Lab rather than Writing Center, so when they
developed the online version—online writing lab—the acronym emerged." However, as Ladv
Falls Brown also explains, "The L in OWL doesn't quite fit online writing center." Labs are
typically associated with remedial work and current-traditional practices, something writing
centers—especially the Texas Tech University Writing Center—try to avoid.
assist in meeting these needs; and SLA theories and assumptions that underlie this
project. Due to the intricate relationship among the three issues enumerated above, they
are not presented separately, but they are interwoven with and form part of the discussion
below, in section 1.1. The second section of this chapter examines and discusses current
approaches to ESL/EFL writing instruction that jusfify the implementation of this study.
This topic is addressed in secfion 1.2. The third section of this chapter provides a review
of the pertinent literature that led to conducting this case study research. This third
section is organized in subsections under different subheadings, which will be provided
later.
Several reasons or issues have motivated this study and provide the contextual
framework for this research project: namely, the author's personal experience as both
EFL learner and EFL instructor; ESL/EFL specialists' contentions presented in books and
journal articles throughout the last twenty years, and the inconclusiveness of research
findings in the field of NES and ESL/EFL writing. The first and most compelling concern
relates to my personal experience as an EFL learner, EFL teacher, composition instructor,
and firm believer in the many pedagogical advantages and possibilities that networked
computers can offer when judiciously employed as tools for communication between
human beings. As stated above, these matters, together with the discussion of some SLA
theories, will be addressed under section 1.1. A second issue that contributed to shaping
the research questions that guide this study involves ESL/EFL specialists' and
researchers' speculations or contentions about teaching techniques that actually work, or
could work, in ESL/EFL writing classrooms. These contentions are often based on
writing theories prevalent in the field and will be examined under 1.2. Finally, the third
reason that prompted and gave shape to this study relates to some gaps in the ESL/EFL
writing literature. Research in the revision process, for example, has yielded
inconclusive and sometimes contradictory findings regarding the type of feedback and
the frequency with which students incorporate this feedback when revising their written
assignments. The literature review occupies the last part of this chapter, under section
1.3.
1.1. Background Information: EFL Populations' Special Needs,
SLA Theories and Assumptions, and the Role of Technology
Although exposure to a target language and interaction with its native speakers
constitute only two of the various elements involved in the process of learning a second
or foreign language (L2, FL), these two aspects are crucial variables whose effects upon
language acquisition and development have been widely studied by second language
acquisition (SLA) researchers (ElHs, 1994; Larsen-Freeman, 1991).^ Exposure and
interaction, which can be subsumed under the concepts of "input" and "output" examined
and discussed by researchers such as Kiashen (1982), Long (1981, 1983, 1985, 1996),
Schachter (1986a, 1986b), and Swain (1985), represent two fimdamental factors in
^ Ellis (1994) provides a comprehensive and meticulous account of SLA theories and
research carried out in the last 40 years—a period of time that indicates approximately how old
this field of study is-in The Study of Second Language Acquisition. Oxford: Oxford University
Press. I have chosen to follow Ellis's framework for approaching the field of SLA because he
seems to have done one of the most exhaustive analyses and enlightening interpretations of the
field's main theories and research studies to date. Larsen-Freeman (1991) also provides a helpful
but more succinct overview of the history and main concerns of the field of SLA, in her article
"Second language acquisition research: Staking out the territory," featured in TESOL Quarterly
25(2), 315-350.
ESL/EFL learning. Rod Ellis, a noted SLA researcher and theorist, points out that
"despite...methodological problems, a number of insights regarding the role of input and
interaction have been gained" (Ellis, 1994, p. 278). In spite of these methodological
problems concerning research design, some evidence suggests that learner output and
collaborative discourse facilitate—in not very clearly understood ways—SLA (Ellis, 1994,
pp. 243-291). Input and interaction research has generated L2 pedagogical theories and
practice that promote exposure to the target language and interaction with its speakers as
key elements in the process of SLA. L2 teaching situations in monolingual countries,
however, are often characterized by students' lack of exposure to the target language and
limited language production in meaningful, communicative situations. L2 instructors
work hard to compensate for these limitations by becoming the best models possible for
their students, by incorporating authentic materials in their classes, and by creating an
environment conducive to communicative interaction.
In EFL situations such as those in Argentina and most Latin American countries,
both students and instructors are often eager to get in touch with native speakers of
English because they believe that interaction and communication greatly contribute to the
development of their L2 skills. EFL students, however, have only limited access to the
"real" (i.e., everyday or colloquial) language used by NESs. It is true that these students
have access and often resort to authentic materials such as magazines and books written
by NESs in order to practice the reading skill. It is also true that they rely on movies, TV
shows, radio programs, and various kinds of recordings produced in English-speaking
countries to improve the listening skill. Undoubtedly, both printed materials and audio-
visual aids are valuable pedagogical tools, mainly because these tools allow for exposure
to "comprehensible input" or input that "contains structure that is 'a little beyond'" the
learner's current level of language competence, as Steven Krashen postulates (1982, pp.
20-30)."* Nevertheless, understanding and processing input are not enough if these
activities are not accompanied by the students' actual language production, as Swain
contends (1985). Even though leamers in EFL settings are exposed to the English
language through various media and they also interact and receive input and feedback
from other EFL leamers and instructors, exposure and interaction are limited in the sense
that they are circumscribed to classroom activities in which only foreign speakers—
whether they be students or teachers—participate. This brief description of the contrast
between the prevalent situation in monolingual countries where an L2 is taught and
current SLA theories indicates that something should be done if EFL leamers are
expected to succeed in acquiring and using the English language. Ways of helping EFL
leamers to speed up this process could—and probably should—include providing them
with the means to be able to interact with NESs.
At present, technological advances make it possible to bring EFL leamers in
contact with NESs, even though the two groups of people may remain geographically
'* Krashen explains language acquisition in terms of five hypotheses that make up his
Monitor model. One of these hypotheses is the Input Hypothesis, which he states m three parts:
"(1) The input hypothesis relates to acquisition, not learning. (2) We acquire by understanding
language that contains structure a bit beyond our current level of competence (/ + /). This is done
with the help of context or extralinguistic information. (3) When communication is successful,
when the input is understood and there is enough of it, / + / will be provided automatically"
(Krashen, 1982, pp. 20-22). The formula {i + 7), which Krashen has often used to refer to the
Input Hypothesis, embodies the concept of language input which is "a httle beyond" our language
competence. In this formula, / "represents current competence" and / + / represents the next level
of language competence (Krashen, 1982, pp. 20-21).
thousands of miles apart. In other words, widely accepted SLA theory and pedagogy
advocating exposure and interaction can be implemented via the electronic exchange of
text through the use of computer wide area networks (WANs). In addition to the widely
used tape-recorder and VCR, one more efficient way to expose EFL leamers to the
English language and to encourage them to test their developing language skills with
NESs is provided by networked computers. The exchange of ideas in the shape of
written discourse between EFL leamers and NESs can be easily accomplished by
establishing connections between NNES and NES educational institutions through
WANs.
The various media of communication that the latest technology provides offer a
wide and untapped spectrum of possibihties for the kinds of contacts that allegedly
promote L2 acquisition and/or development. These media include satellite and cable TV,
fiber-optics communication, multimedia software, fax machines, and networked
computers with access to e-mail services, newsgroups, the World Wide Web, and the
like. My main claim throughout this work is that the proper use of CMC offers a myriad
of pedagogical potentials in EFL settings since electronic communication and the
exchange of written text can give EFL students in monolingual countries opportunities
for input from and interaction with NESs that they would not have otherwise. A few of
the many valuable opportunities that CMC provides that would contribute to developing
EFL learners' writing skills include, for example, giving leamers the chance to put their
emerging language skills to the test, to develop their often limited vocabulary, and to
improve their grammar and discourse organization in English.
The principles of input, interaction, and meaningful communication mentioned
above constitute the backbone of the Monitor Model (Krashen, 1981, 1985) and its
application in the classroom (Krashen, 1982; Krashen & Terrell, 1983). Rod Elhs warns
us, however, that Krashen's Monitor Model has been challenged and criticized on various
grounds by other SLA researchers such as Tarone, McLaughlin, Gregg, and Ellis. (Ellis,
1990, pp. 59, 93-129; 1994, pp. 356-366, 612-663). Yet, Ellis also admits to "the
lucidity, simplicity and explanatory power of Krashen's theory" (1990, p. 57). In effect,
the pedagogical model advanced by Stephen Krashen currently guides much classroom
EFL/ESL methodology and provides a point of departure for my project.
Krashen explains and emphasizes the potentials that the language classroom has
to offer, especially for leamers at the beginner level. The language classroom provides
what he calls "comprehensible input" or "input that is optimal for acquisition" (Krashen,
1982, p. 58). He draws a distinction, however, between the types of students who can
benefit from the informal environment provided by the real world and those who would
benefit more from the formal classroom environment:
Despite my enthusiasm for the second language classroom, there are several
ways in which the outside world clearly excels (or some "modification" of the
outside world...), especially for the intermediate level second language student.
First, it is very clear that the outside world can supply more input. Living in the
country where the language is spoken can result in an all-day second language
lesson!. . . The informal environment w i l l . . . be of more and more use as the
acquirer progresses and can understand more and more. (Krashen, 1982, p. 59)
I would go fijrther and suggest that intermediate and advanced EFL students—those who
have already an average-to-good mastery of the four language skills—would benefit
greatly from the interaction with native speakers of the target language, not only because
they would be able to practice their language skills while receiving feedback from NESs,
but also because being aware of some of the intricacies of leaming a second language,
they are in a position to talk about the language, that is, to make a meta-analysis of both
their language production (what they speak and write) and their language reception (what
they listen to and read).
Many ESL/EFL specialists agree that situations in which students are required to
make themselves understood—whether it be in the spoken or written medium—so that
communication can actually occur provide an excellent environment for leaming a
foreign language. In E-mail for English Teaching: Bringing the Internet and computer
leaming networks into the language classroom, Mark Warschauer summarizes some of
the reasons why the use of Intemet resources such as e-mail should prove advantageous
for those people leaming an L2. In his introduction to this book and when referring to
ESL students in the USA, Warschauer states:
. . . e-mail provides students an excellent opportunity for real, natural
communication. Many of our students—often those living or smdying in English
speaking countries—lack sufficient opportunities for communicating in English.
E-mail can put students in contact with native speakers and/or other English
leamers across town or around the world in minutes and provide the authentic
contexts and motivations for communication that teachers are always trying to
supply. (1994, p. 2)
If, as Warschauer contends, communication through e-mail proves beneficial for those
people who are studying English in English-speaking communities, I contend that this
type of communication should be much more beneficial for EFL students in a foreign
country than for ESL students in an English-speaking country, since the former have
limited opportunities to interact with native speakers of English.
For this dissertation project, therefore, I decided to investigate various ways in
which the use of CMC, more specifically the use of e-mail, can benefit advanced EFL
students who are trying to become proficient in their use of English in a South American
country. This qualitative classroom study focuses, however, only on one of the four
language skills, writing. Furthermore, as stated above, this research study is restricted to
the examination of three main issues or questions: (1) What kind of feedback EFL
students' writing tends to receive when these students send their short essays via e-mail to
NESs working for an on-line writing center at a university in the USA; (2) What kinds of
changes in EFL students' drafts—if any—are related to the NESs' suggestions for
improvement; and (3) What are EFL student writers' perceptions about that feedback
from and interaction with native speakers.
Before current theories and practices in ESL/EFL writing instruction are
discussed and before the literature pertinent to EFL writing and CMC is reviewed, it is
necessary to examine a commonplace concept in SLA circles, i.e., the notion of
difference between ESL and EFL leamers, in order to account for my contention that
EFL leamers need more input from and interaction with NESs than do their ESL
counterparts in their journey towards second language development and proficiency. In
the domain of L2 leaming, several groups of leamers have been identified. These
language leamers are grouped and labeled according to the purposes they have for
leaming a foreign language and according to the situation they often find themselves in.
Two of the groups that have been identified as special groups are ESL and EFL leamers.
Even though ESL and EFL leamers share certain similarities in their process of acquiring
10
a language other than their native one, these two groups often study English for different
purposes and are characterized by particular circumstances. ESL students often have to
leam English out of necessity, because they need English for survival, as a vital means of
communication to be able fiinction in the society where they live. Two examples of
people in ESL situations are the inhabitants of India, who need to leam and use English
because Enghsh is one of the official languages of that country, and people who
immigrate to the USA and need to leam English to be able to interact with NESs. EFL
students, on the other hand, usually study English for more particular and less vital
purposes. For example, they may study English to conduct business transactions, to go on
a trip abroad, to be able to read journal articles in their academic field of study, or just for
the fim of it.^
More important than the apparent differences in the purposes for studying English
that these two groups present, however, are the different circumstances in which ESL and
EFL students leam the target language. ESL leamers are often in contact with the target
language on a daily basis and when performing everyday activities, while EFL leamers
are limited in their access to and practice of the target language. The former have
obviously far more opportunities of actually using the language than the latter. Most of
the time, EFL students can only listen, speak, read, and write with and among people who
are also non-native speakers of English in a formal environment, the classroom. In other
words, interaction in the target language takes place mostly between teachers and
^ This information about the different kinds of second language leamers and the kinds of
environments they find themselves in is commonplace knowledge among ESL/EFL specialists,
but a good and comprehensive account of the social contexts and environmental factors
characterizing various groups of L2 leamers can be found in Elhs (1994, pp. 214-229).
11
students or among students, most of whom are usually non-native English speakers. If
we accept current SLA theories that postulate interaction and meaningful communication
as the best ways to leam a second or foreign language, we would agree that the more
instances non-native speakers have to be exposed to the target language and to interact—
whether it be via oral communication or written discourse—with native speakers, the
sooner they will acquire or become more proficient in their use of the target language.
The above situation is best illustrated through a personal narrative. By means of
the following scenario, I hope to demonstrate the limited possibilities EFL students have
of interacting with NESs and also to summarize the circumstances that prompted me to
study how EFL instmctors, with the help of available technology, can make interaction
with NESs a finitfiil experience. I am an EFL instructor, but before obtaining my
teaching degree in Argentina, I was an EFL student myself. As many children and
teenagers do in monolingual countries such as Japan or Italy, I attended English language
classes at a private school twice a week for approximately six years. Not only while
attending these language schools or institutes but also later, upon becoming university
students in an EFL/English Literature program, my classmates and I were always
seeking opportunities to practice, outside the classroom, what we were leaming about the
English language. In more technical terms, we wanted to test our hypotheses about the
rules of English (Elhs, 1994, p. 30). We tried to do so by Hstening to tapes recorded in
England or the US, by becoming pen pals with NESs, by receiving English-speaking
guests in our homes, and also by travelling to England, the US, Australia, or New
Zealand—if we happened to have the means to do so. Our desires and hopes to interact
12
with NESs stemmed from the inner need to try our skills against the real world, the native
speakers of English. In other words, if we were going to become EFL teachers, models
for our fiiture students, we needed to put our competence and performance to a test, not
just by communicating with our non-native Enghsh teachers or classmates, but also by
interacting with NESs. Unfortunately, very few of us were able to test our English
language skills in this fashion, and, if we did so, it was only on rare occasions.
Once in a while we would have the chance of talking to a NES, for example,
when tourists or exchange students from the United Kingdom, the USA, Australia, or
Nev/ Zealand would visit our home town. Other occasions for interaction with native
speakers occurred every time EFL/ESL speciahsts from England or the USA would
participate in a conference or give lecmres. However, on these rare occasions we would
be limited to the activity of listening. In other words, fluent oral interaction or dialogue
with these NESs was not always possible. Another possibility for interaction consisted in
establishing correspondence with NESs abroad. However, it was not always easy to
obtain a pen pal to write to, and even after obtaining one, the postal service tended to be
slow, and therefore back and forth correspondence tended to take too long and in many
cases became interrupted and never resumed. These are only a few examples that
illustrate the limited opportunities EFL students have—or rather used to have about ten or
fifteen years ago—to practice their language abilities and to interact with NESs. The
situation is changing, though, especially due to the recent advances in computer and
communication technology.
13
Upon becoming a university smdent in the US, I had to re-adjust my aural and
oral skills to new patterns of pronunciation and intonation that I was aware of but I had
not tried out in the artificial situation provided by the classroom in my home country. As
far as the writing skill is concerned, I was faced with a number of issues related to
rhetorical concerns that I had been barely aware of before. Although the concept of
revision was not new to me—because I had applied revision techniques without having
the technical labels for them, i.e., "editing," "macro" and "micro-structural changes,"
"rhetorical considerations," etc.—it acquired a new meaning or dimension when I leamed
the whole thrust of the process approach to writing. Through the different readings I did
for various graduate courses I took, I came to the realization that even though structural
elements should not be disregarded when working on a piece of writing, they should not
constitute the first priority during the revision stage. Content and development, together
with the rhetorical considerations of audience and purpose, become more important when
compared to local issues such as sentence structure and the mechanics of a written text.
These are the tenets of the process approach, which only recently have begun to be
considered in EFL classrooms around the world (Leki, 1991a, pp. 203-4).
Both my personal experience as an EFL leamer and the work of many researchers
provide evidence to support the fact that people can leam a foreign language in formal
environments like classroom settings, without actually having to interact with the
speakers of the target language. In their review and analysis of SLA classroom research,
Elhs (1990, 1994), Nunan (1991), and Widdowson (1990), among others, illustrate the
fact that an L2 can be leamed in a classroom. Innumerable articles published in joumals
14
such as The Modem Language Joumal, Studies in Second Language Acquisition, Second
Language Leaming, and TESOL Ouarterlv also attest to this assertion. Furthermore, the
admission and presence of many intemational students in both undergraduate and
graduate US university programs strengthen this claim since probably most of these
foreign students had only formal ESL/EFL instmction and only a few opportunities of
interacting with NESs in their home coimtries before coming to the US. Likewise,
many EFL instructors like me can attest to this contention since they became EFL
teachers without ever leaving their country of origin. In other words, with the right
amounts of motivation and enthusiasm, with devotion, and—undeniably—with some
determination and effort, people can become fluent speakers of a target language.
Nevertheless, and in spite of all these arguments, we caimot overlook the benefits that
contacts with the target language, people, and culture may bring about, whether these
benefits be realized directly or indirectly—through motivational forces—in the actual
improvement of language skills.
As Elhs's analyses (1990, 1994) reveal, the issues addressed by Krashen's Input
Hypothesis (1982, 1985),^ together with the subsequent modifications to it provided by
the Interaction Hypothesis (Long, 1983), the Output Hypothesis (Swain, 1985) and
Collaborative Discourse Hypothesis (Hatch 1978, 1992), relate mostly to oral
communication. Most of the research conducted to test these hypotheses investigates the
^ Krashen's Input Hypothesis is one of the five hypotheses that make up his second
language acquisition theory. The other four hypotheses that Krashen includes and explains m
Principles and practice in second language acquisition (1982) are the Acquisition-Learning
Distinction, the Natural Order Hypothesis, the Monitor Hypothesis, and the Affective Filter
Hypothesis.
15
acquisition and development of phonological, morphological, and syntactical structures
during spoken interaction. I extrapolate these hypotheses to written input and written
interaction, for two main reasons. On the one hand, I envision their logical and
ftinctional applicability to written discourse exchanged between people communicating
via computer networks. This study, therefore, investigated the possible development of
the writing skill of intermediate-to-advanced EFL students at a teacher-training college in
a monolingual country, when they were given the opportunity to correspond with NESs;
i.e., when they were given the possibility to produce some writing (output) and receive
some feedback (input), in a meaningfiil way. On the other hand, the aims and prevailing
characteristics of the EFL leamers participating in this study, namely prospective EFL
teachers, coincide with the basic premises of the above hypotheses. In other words, the
EFL leamers who participated in this project needed to understand input in the English
language, wished to try out their knowledge of English against a real audience of NESs,
and, in most cases, wanted to produce and use the Enghsh language as proficiently as
possible so that they eventually would be able to transmit it to other EFL leamers.
1.2. Current Approaches to ESL/EFL Writing Instruction that
Justify the Implementation of this Proiect
My contention that EFL populations would benefit from interaction with and
feedback from NESs is not restricted to the speaking and hstening skills; instead, it
encompasses the four language skills involved in SLA, namely, listening and reading
(receptive skills), and speaking and writing (productive skills). However, this work will
focus specifically on the writing skills of EFL leamers for two main reasons. On the one
16
hand, writing is no longer a neglected skill in the ESL/EFL curriculum, a skill relegated
to the reinforcement and practice of grammatical structures, as it used to be around the
middle of the twentieth century (Leki, 1992; Reid, 1993). ESL/EFL writing methodology
has undergone a change from the early days of the audio-lingual method (approximately
four decades ago), with its emphasis on the oral-aural approach, to a more social and
discourse-oriented pedagogy that values and fosters the personal expression of ideas, the
careful consideration of audience and purpose when writing, and the tenets of the process
approach (Leki, 1992; Raimes, 1991; Reid, 1993; Silva, 1990). Furthermore, in the past
twenty to thirty years, there has been an increasing interest in various aspects of ESL
writing, not only on the part of ESL instructors and composition teachers of NESs but
also on the part of researchers, as many ESL/EFL writing specialists indicate (Silva,
1990; Kroll, 1990; Leki, 1992; Reid, 1993). This recent increased attention to writing in
the ESL/EFL curriculum in educational institutions in the US and abroad has produced a
considerable amount of research. In spite of the numerous studies concerning the process
involved in ESL/EFL writing, Tony Silva, a well-known ESL/EFL writing specialist,
cautions us that most of them have been exploratory in nature, methodologically flawed,
and ultimately inconclusive (1990, p. 20). Consequently, we should take these research
results and conclusions with circumspection and avoid making hasty generalizations that
do not necessarily hold true in all ESL/EFL situations. Echoing Silva's reflections, I
maintain that the current state of affairs calls for fiirther research that may support,
modify, or question particular methodological practices in particular settings for
particular populations.
17
On the other hand, another reason why this work revolves around the
development of only the writing skill in EFL leamers is that, among the four language
skills, writing is the best suited for a study that incorporates computer networks as a
variable. Computers lend themselves naturally as an excellent medium for written
communication or written discourse. Stated more emphatically, the use of computers and
CMC offers what I believe to be one of the most practical, convenient, and efficient
media for EFL students-especially if they already have access to technology-to practice
their developing interlanguage system, to put their vocabulary repertoire to a test, and
more importantly, to develop their writing skills in an interactive and process-oriented
fashion. Although the act of written communication involves also reading skills, since it
is through reading that students can obtain input from NESs, the reading skill does not
constitute the focus of this work. The speaking and listening skills will not be part of this
discussion either since CMC, in the form and to the extent that the EFL subjects
participating in this study had access to it, did not allow for oral-aural interaction with
NESs.
Finally, within writing—the skill under scmtiny here—I have chosen to foreground
the revision stage. My decision to do so resuks from a number of conclusions I have
arrived at after reviewing the ESL/EFL research. First, the revision stage seems to
occupy a preeminent place in writing methodologies such as the process approach and
writing for special or academic purposes approach, which currently constitute two of the
most favored writing teaching methodologies. Both these approaches—without
neglecting the invention and editing stages, but favoring different teaching
18
methodologies—insist on the importance of reworking, retouching, reformulating,
reorganizing the written text so that the final product achieves its intended effect on the
audience to whom it is addressed. Second, the handling of feedback and the
incorporation of revisions in a text can be more easily accomplished through the use of
word-processors than through the use of pen or pencil and hand-writing. Various
computer fimctions such as "copying," "pasting," "deleting," and the printing of texts
facilitate the revision process and, in a way, make revision a natural and logical stage in
the writing process. Last, but not least, another reason for focusing on the revision stage
concems the fact that detecting, measuring or rating revision changes can be more
accessible than detecting, measuring or rating invention processes, for example, since the
activities involved in the latter often remain hidden to the researcher in the subject's
"black box." In other words, the latter are seldom completely materialized in the shape
of text on a computer screen or a sheet of paper and, hence, are also more difficult to
trace back.
Some observations made by ESL writing specialists should be considered at this
point because they provide a rationale for conducting this research. One such reflection
concems the persona of the writing teacher. In the article "Coaching from the margins:
issues in written response," Ilona Leki (1990) raises a fundamental question related to
the task of responding to students' essays, i.e., the complex role played by the teacher.
This issue has been discussed by many composition specialists in the field of NES
writing, among them Knoblauch and Brannon (1981, 1984). Leki, however, adds to the
discussion by characterizing the role of the writing teacher as "schizophrenic, split into
19
three incompatible personas: teacher as real reader (i.e., audience), teacher as coach, and
teacher as evaluator" (p. 59). Leki sees this sptit as a negative aspect in the revision
process and to support her point, she explains:
Some educators advocate responding to student writing as real readers, not as
writing teachers, questioning or commenting on the writing only when we might
genuinely do so if we were reading a published text. But given the unequal power
inherent in the roles of teacher and student, it is umealistic to pretend that teachers
can read student texts in the same way as we read texts we select for ourselves.
Furthermore, since the role of the writing teacher is to teach writing, the teacher is
constantly forced away from the content of the text toward the way the content is
presented. L2 students in particular expect and require greater intervention than
that of a real reader....
An even more profound schizophrenic split that vmting teachers, particularly
process-oriented writing teachers, experience is that of trying to be at the same
time the coach and the evaluator of student writing. For example, if a teacher has
collaborated with students rather than teach/evaluate, and the students' work is
then judged insufficient by the standards of that educational setting, the teacher
has, in a sense, betrayed the students by not intervening more heavily, (pp. 59-60)
It is precisely the tension of this dualism that Leki refers to in the roles of "teacher" and
"real reader," and "coach" and "evaluator" that can be completely avoided or at least
partially relieved by allowing writing center consultants to participate in the revision
process and to assume the roles of real readers and coaches. It is the chance of working
with on-line writing center consultants that could provide students with the possibility of
obtaining the opinion of independent and disinterested readers, readers who are
presumably willing to contribute their perspective or reaction to a piece of writing
without the pressure that having to assign a grade would entail.
It is common knowledge among ESL/EFL instmctors and researchers that there
has been a movement or evolution from the stiff parameters of the early grammar
20
translation and the audio-hngual methods to the present-day emphasis on the meaningful
communication. The earlier methodology—often considered obsolete but still practiced at
present—emphasized memorization and drilling of structures, in most of the cases at the
expense of meaning. The most recent interest in ESL/EFL instruction, however, falls on
meaningful commimication rather than on formal accuracy and correctness. This interest
becomes apparent in the predominance of notional-functional syllabi and in the scant
attention paid to grammar rules and explanations in textbooks (Raimes, 1983).
This evolutionary movement in ESL/EFL instruction has affected the teaching of
the four skills, i.e., speaking, listening, reading, and writing, to various degrees. The
concomitant but gradual transformation in the techniques for teaching writing is reflected
in the current emphasis placed on "communication" rather than on "accuracy." Content
and communication of meaning have become, at least in theory, priority number one in
the teaching of composition to ESL/EFL leamers. I say "in theory" because the concepts
encompassed by the label "process"—which has actually become a buzzword, in
ESL/EFL circles—are not frequently put into practice. Perhaps because many of today's
ESL/EFL instructors are the sons and daughters of the audio-lingual method, we are still
reluctant to let go of the grammar-based syllabi and textbooks and tend to focus our
' In her article "Tradition and revolution in ESL teaching," however, Raimes emphasizes
that this tendency or shift towards communicative competence away from drill-driven teaching
methodologies does not yet constitute a new paradigm in the field. Relying on Thomas Kuhn's
observations in The stmcture of scientific revolutions (1970), Raimes explains that although
recent syllabi design, textbooks' content, classroom practices, and research reflect some
movement towards "the new unknown territory," they still encompass traditional views of
language teaching and leaming. She states: "...ad hoc modifications [such as the inclusion of
functions in textbooks and classroom practices that incorporate "opportunities for interaction"]
are abundant in our field, providing evidence of a tentative questioning of tradition and yet
reflecting a basic distmst and fear of abandoning it completely" (Raimes, 1983, p. 541).
21
attention on formal aspects of writing rather than on content, organization, and rhetorical
concems.
In an attempt to incorporate these latest trends in composition instruction into an
EFL writing environment and in order to place the roles of "real reader" and "coach" in
one person, and that of "evaluator" in a different person, I decided to embark on this
project. This case study, therefore, focuses on revision rather than on invention or
editing, as a means of allowing the students to explore ways in which they can
communicate their ideas more effectively. One appropriate application of the new tenets
in EFL writing instmction, I thought, would be having EFL students in distant places
communicate with and receive feedback from readers who believe in and are deeply
involved with the aims of the process approach. These readers, consultants, or tutors,
who work in an on-line writing center, could be accessed by the use of electronic mail.
These consultants became the most logical, convenient, and suitable choice as a source of
help for advanced EFL students engaged in revising their written work, for two main
reasons.
On the one hand, on-line writing center consultants often have as a premise to
respond to a piece of writing globally, without stopping at every single formal or
structural problem they come across. This philosophy, which sees the writing center as a
sounding board for students' ideas, as a dialogic instrument, a place where students can
seek and find encouragement and guidance for their work in progress, is clearly
delineated in numerous pubhcations (Carino 1992, 1995a, 1995b, 1996^ Ede, 1989;
In a recent article, "Open admissions and the construction of writing center history: A tale
22
Lunsford, 1991; North 1984) and daily attested by the on-line discussions held by writing
center consultants and other subscribers to the listserv <wcenter@listserv.ttu.edu>.
Consequently, it was assumed that, following these principles or this philosophy, on-line
writing center consultants would enact the role of coaches, guides, and real readers,
especially because they would provide suggestions and advice without having to provide
a grade as well. On the other hand, EFL students could obtain assistance from readers
other than their instructor, i.e., from NESs working for on-line writing centers, without
actually having to leave their country for this type of help.
The writing center that generously and disinterestedly made its services available
for this project, the Texas Tech University On-line Writing Center, sees its primary aim
as helping people to become better writers and assisting them in leaming how to revise a
piece of writing on their owoi. As expressed in the "Frequently Asked Questions" section
in the Texas Tech University Web Page, it is not the Writing Center's job to edit
students' written assignments or to assure them that they will obtain a good grade:
Does the UWC provide editing services?
No. Consultants at the University Writing Center do work with grammar issues,
but our goal is to help our clients to become better writers. Therefore, we prefer to
help writers leam how to identify their own grammar errors, rather than simply
editing papers for them....
of three models," Peter Carino (1996) recognizes and describes two attempts of the writing center
community to define itself historically. Carino calls these two attempts "the evolutionary model," and
"the dialectic model." "The evolutionary model of writing center history," Carino explains, "cites the
open admissions initiatives...as a watershed moment when centers proliferated as remedial clinics and
labs and then evolved into the full service centers of today" (p. 31). Also according to Carino, however,
the other model "maintains that these centers immediately, or quickly, rejected their imposed roles as
course supplements responsible for remedial grammar and developed an innovative student-centered
writing pedagogy that competed with classroom work" (p. 31). As an alternative to these two models,
Carino proposes the cultural model which, without subsuming the other two, "allows for both" (p. 31).
More importantly, the cultural model that he advances is aware of the historical narrative that it constmcts, recognizes the limitations of "representing history in language," and also accepts the need and
challenge of "thick descriptions of the multiple forces impacting writing centers" (p. 30).
23
Can the UWC guarantee a particular grade on an assignment?
No. Writing consultants are not evaluators. Instead, we try to act as readers,
providing audience feedback for purposes of revision. (Texas Tech University
Writing Center, http://english.ttu.edu/uwc/owl/FAQ.html)
The answers to these two questions reveal the type of assistance writing center
consultants can provide people interested in having their written text read by an
independent audience. In addition, the Texas Tech University Writing Center has a firm
commitment to assisting all types of writers. Their mission statement clearly articulates
the Texas Tech University Writing Center's three main objectives:
- to assist writers, regardless of their status (faculty or students, undergraduate or
graduate), their level of proficiency, or their college, during the various stages of
their writing project;
- to create a supportive environment in which writers and their readers can work
effectively one-to-one, both face-to-face as well as onscreen and online; and
- to train writing consultants to become knowledgeable, effective readers of and
responders to texts from various disciplines.
(Texas Tech University Writing Center,
http://english.ttu.edu/uwc/owl/mission.html)
In view of the above information, therefore, not only the easy accessibility, but also the
aims of the Texas Tech University On-line Writing Center made it reasonable, feasible,
and appropriate to request the participation of on-line writing center consultants in this
research.
1.3. Literature Review
In this third section of the chapter, the main lines of research that have been
followed in the fields of ESL/EFL writing and CMC are outlined. A careful examination
of recent research in these areas of knowledge reveals some gaps that should be bridged
24
by specific research. Two aspects that need more in-depth study, not only because they
have not received enough attention from researchers, but also because of their alleged
benefits and pedagogical usefulness are the interaction between NES and ESL/EFL
leamers and the feedback that NESs can offer ESL/EFL student writers. The following
two subsections, therefore, summarize major research findings and point out some theoretical issues conceming ESL/EFL writing (1.3.1), and CMC in NES and ESL/EFL
settings (1.3.2), as a way of laying the foundations for this research. When addressing
issues related to ESL/EFL writing in section (1.3.1) and also when discussing CMC in
ESL/EFL environments in section (1.3.2), I specifically mention research problems that
need further study. When examining CMC in NES settings, however, I do not point out
gaps in the research since this particular field is not the focus of my dissertation. In other
words, I only highlight issues of concern to the field of CMC and NES writing such as
text production, collaboration, ease of communication, teacher-student and studentstudent relationships, and some others, which could directly or indirectly relate to
ESL/EFL situations. Most—if not all—of these issues seem relevant to any group of
students using CMC, and, therefore, I examined them prior to conducting my research in
CMC in an EFL setting.
1.3.1. ESL/EFL Writing
In the past, research and classroom methodologies in the field of ESL/EFL
writing were mostly informed by findings in linguistics. More recently, however,
ESL/EFL writing instmction has rehed, with increasing frequency, on research findings
25
and pedagogical theories prevalent in Enghsh composition studies, that is, the field of
wnting instruction for NESs. In this journey from linguistic-based theory and practice to
a type of pedagogy which often embraces process approaches and collaborative strategies
to teaching, ESL/EFL writing theory and practice continue to evolve, moving further and
further away from the audio-lingual method prevalent between 1940 and 1970 (Reid,
1993, p. 22).
ESL/EFL speciahsts such as Tony Silva (1990), Ann Raimes (1991), Ilona Leki
(1992), and Joy M. Reid (1993) have provided comprehensive historical accounts of
approaches to ESL writing instmction. They have explained the main principles that
characterize and guide what they consider to be the most widely used methods in the
teaching of ESL writing to date. These four authors classify the main approaches in a
similar fashion but attach different labels to them. In his review of ESL/EFL writing
pedagogy, Silva includes four movements: "controlled composition," "current-traditional
rhetoric," "the process approach," and "Enghsh for academic purposes" (1990). The
"controlled composition" approach relies on imitation of model paragraphs and
emphasizes accuracy and correctness of form rather than communication of meaning.
Within the parameters of this approach, "the text becomes a collection of sentence
pattems and vocabulary items—a linguistic artifact, a vehicle for language practice. The
writing context is the ESL classroom; there is negligible concern for audience or
purpose" (Silva, 1990, p. 13). The "current traditional" approach derives from current
traditional methodology in NES composition instruction and the theory of contrasti\e
rhetoric postulated by Kaplan (1966, 1967). According to Silva (1990), "The central
26
concern of this approach [is] the logical construction and arrangement of discourse
forms" (p. 14). The paragraph with its constituent elements (e.g., topic and supporting
sentences) and the various techniques to develop it (e.g., definition, classification,
exemphfication, etc.) receive most of the attention. One of the main goals of this
approach is to help leamers to master the traditional modes of writing: descriptive,
narrative, expository, and argumentative writing.
In reaction to the two previous approaches, which prioritize structural elements at
the sentence and paragraph levels, the "process approach" emphasizes the non-linearity
and recursiveness of writing, the discovery of meaning during the act of writing, and the
inter-relatedness of all the stages of writing, i.e., invention and planning, drafting,
revising, and editing. As Silva states when summarizing the main thrust of the process
approach: "The writer is the center of attention—someone engaged in the discovery and
expression of meaning; the reader focusing on content, ideas, and the negotiating of
meaning, is not preoccupied with form. The text is a product—a derivative concem
whose form is a function of its content and purpose" (Silva, 1990, p. 16). Finally,
supporters of the "English for academic purposes approach" react against the tenets of the
process approach on the grounds that the latter does not prepare students for the
expectations and demands of academic writing. Supporters of this approach believe that
students should be socialized according to the parameters of the academic community
they aim to become members of; writing should be taught so that the students are made
aware of the genre and discourse pattems favored by the academic community they will
belong to one day. Therefore, teaching practices focus both on having students read,
27
examine, and analyze sources and discourse forms typical of the students' respective
fields of interest and on having the students present their written work according to what
is considered customary and acceptable discourse in target fields.
Although Raimes (1991) points out and describes similar techniques and
pedagogical goals and although she often mentions the same approaches discussed by
Silva, she classifies the main trends in ESL writing according to the aspect of writing that
has been emphasized by its practitioners, i.e., the form, the writer, the content, and the
reader. Most of the methodologies employed in the ESL classroom have been influenced
by, and to a certain extent have resuhed from, pedagogical approaches common in the
field of NES writing (Silva, 1990, p. 11; Reid, 1993, pp. 21-47). Silva argues that all
these approaches, which are still followed in varying degrees by ESL instmctors today,
have had a fluctuating history, going from trial practices, to acceptance, to criticism, to
rejection, to dormancy, and back to new trial practices. He observes that although these
four approaches have meant progress "in understanding the phenomenon of L2 writing,"
they are also indicative of a lack of consensus in the field and absence of a coherent body
of knowledge (Silva, 1990, p. 18). He therefore emphasizes the need to improve this
situation that "engenders a great deal of confusion and insecurity among ESL composition teachers" (p. 18) and offers some suggestions to modify the present situation.
The one recommendation most relevant to this study states that composition
instructors need to ground any ESL/EFL writing approach in "adequate theory" and
"credible research." Expanding on this idea, he notes that in spite of the growing body of
empirical research, most of the studies in the field:
28
(1) Are primarily small-scale and exploratory; (2) are somewhat uneven in quality
and diffiise in terms of focus, methodology and orientation—and thus are not
easily compared or synthesized; and (3) often have not been conceived, conducted
or interpreted within adequate models ofL2 composition [my emphasis]. Finally,
published research on the relative effectiveness of different approaches when
applied in the classroom is nearly non-existent. (Silva, 1990, p. 20)
Even though Silva's observations were made some years ago and thus could be considered somewhat outdated, they still apply to most of the research conducted in ESL/EFL
writing to date. A considerable number of studies have been carried out during these
eight years. Most of these studies, however, seem to be still well characterized by Silva's
statements. His reflections clearly reveal certain limitations in the ESL/EFL writing
research conducted so far and, at the same time, call for a type of research that is based
on L2 writing frameworks.
Silva (1990)^ and otiier ESL/EFL writing speciahsts, e.g., Leki (1992) and Reid
(1993, 1994), have also pointed out significant differences between ESL/EFL writers and
their NES counterparts and, at the same time, have stressed the need to attend to the
special nature of ESL/EFL writers both in teaching situations and when conducting
research. Moreover, quite a few studies have investigated and reported differences
between ESL/EFL and NES populations as far as writing is concemed. Silva offers a
review of empirical research that compares ESL/EFL to NES writing processes, and he
underscores the need to treat ESL/EFL writers' concems differently from those of NES
writers (Silva 1993, 1997). This need to treat and study ESL/EFL student writers
^ In his later work, Silva continues with the same line of reasoning, stressing the need to
keep a clear distinction between the nature of ESL/EFL and NES writers and writing processes
(see for example "Toward an understanding of the distinct nature of L2 writing: The ESL
research and its implications" in TESOL Quarterly 27(4). 657-677, 1993; "On the ethical
treatment of ESL wnters" in TESOL Quarterly 3 U2), 359-363, 1997).
29
differently from the way in which NES student writers are often considered constitutes
one of the reasons that prompted this study. One of the premises or hypotheses on which
this study was based was that a focus on a group of Spanish-speaking advanced EFL
leamers would allow the researcher to assess the particular characteristics and needs of
the EFL population participating in this project and to arrive at conclusions that pertain to
this particular group of students at this particular educational institution.
Quite a few studies on the processes ESL/EFL students go through when writing
have been conducted and subsequently featured in renowned joumals such as TESOL
Ouarterlv, The Joumal of Second Language Writing, and Studies in Second Language
Acquisition. In the area of revision (v/hich seems to be one of the aspects most widely
studied), issues such as peer and teacher feedback have been researched with varying
results. Some of the studies have focused on the written feedback from peers and
instructors on students' essays. Within this group of studies, quite a few have paid
particular attention to learners' attitudes and reactions to feedback (Ferris, 1995, 1997;
Ferris, Pezone, Tate, & Tinti, 1997; Hedgcock & Lefkowitz, 1994; Jacobs, Curtis, Braine,
& Huang, 1998; Mangelsdorf, 1992; Mendonca, & Johnson, 1994; Murau, 1993; Nelson
& Murphy, 1992, 1993; Zhang, 1995); others have focused on the students' processing
and handling of feedback (Cohen, 1987, 1991, Cohen & Cavalcanti, 1987, 1990; Porte,
1996). Moreover, the focus of additional studies has been textual features that remain the
same or change from draft to draft (Connor & Asenavage, 1994; Ferris, 1997; Goring
Kepner, 1991). Still other studies have investigated the characteristics and effects of
feedback provided by ESL teachers (Goldstein & Comad, 1990; Ferris, Pezone, Tate, &
30
Tinti, 1997; Ihde, 1994; Zamel, 1985) andtiie characteristics and effects of feedback
provided by peers (Caulk, 1994; Devenney, 1989; Lockhart and Ng, 1995; Mangelsdorf,
1992; Murau, 1993; Obah, 1993). More theoretical or philosophical issues have also, but
less frequently, been addressed. For example, questions such as the appropriation of
students' texts (Reid, 1994) and the transmission of US institutions' cultural values to
ESL writing classrooms have received some attention lately (Auerbach, 1991;
Canagarajah, 1993; Santos, 1992; Zamel, 1997).
Even though I have pointed out what appear to be the main emphases conceming
the revision stage in ESL/EFL situations in the above studies, there are often no clear-cut
divisions as to the aspects examined by researchers. In many cases, although the purpose
of a study may revolve around the students' perceptions and reactions to peer feedback,
for example, some researchers may also evaluate data and arrive at conclusions that offer
new insights into a related aspect such as the texts produced by the students after they
have received suggestions. In other words, these and other studies have produced
interesting results that merit discussion and serve as bases for further research, but their
conclusions cannot be applied or extrapolated to particular populations such as the one
which constitutes the focus of this study.
Furthermore, most of the studies listed above have been carried out in regular
classes where either ESL/EFL instmctors or peers provide some type of feedback or
respond to drafts. Overall, these studies suggest that feedback is perceived as beneficial
by ESL/EFL leamers and it often produces some effect on students' drafts (sometimes
negligible, most often positive, and less frequently negative), especially if the feedback
31
comes from teachers. None of the aforementioned studies, however, has explored EFL
students' reactions to on-line writing center tutors' assistance or the revision of their texts
upon receiving help via electronic mail. Therefore, two aspects that are addressed in this
study are Spanish-speaking EFL learners' attitudes and perceptions towards comments
and suggestions from NESs about their writing and also the types of feedback provided
by these independent NES tutors—readers other than the classroom teacher—to these EFL
writers.
As far as unpublished works are concemed, topics such as ESL/EFL writing
processes in general, peer and teacher feedback, and revision strategies have been
explored in a considerable number of dissertations (e.g., Adipattaranun, 1992; Chitrapu,
1996; Caissie, 1997; Dessner, 1991; Malicka, 1996; Mukheijee, 1996; Riley, 1995;
Singer, 1994; Victori Blaya, 1995; Yin, 1995). For her doctoral dissertation, Victori
Blaya (1995) collected data "from 200 undergraduate Spanish students enrolled in EFL
classes at the Universitat de Barcelona... .Case studies were then undertaken with four
subjects (two good writers and two poor writers) who were interviewed and required to
think-aloud [sic] as they wrote an argumentative essay." This dissertation seems to be
one of the very few—or the only one-that deals with the writing process of EFL leamers,
who are native speakers of Spanish. It focuses on differences in EFL students' writing
skills relative to "language proficiency, writing strategies, and Metacognitive Knowledge
(MK) about writing" (Victori Blaya, 1995). Most of the unpubhshed works, however,
have been carried out with ESL/EFL students whose native language is other than
Spanish. Examples of these unpublished works are Jung (1993), with Korean ESL
32
students; Wu (1991) and Lee (1993) with Chinese students; Aly (1992), Asiri (1997), and
El-Shafie (1990) with Middle Eastem ESL students; Mooko (1996) with students from
Botswana; and Yahya (1994) with multilingual ESL students in Malaysia.
Furthermore, it would appear that studies exploring how the reception of feedback
from NES writing center consuUants affects the development of ESL/EFL students'
writing are non-existent or extremely obscure. The only unpublished work on the role of
writing centers in the writing of ESL/EFL students appears to be Carter-Tod's doctoral
dissertation (1995). This unpublished work examines four L2 students and the writing
center tutors who assisted them during one semester, and it includes a survey of the
writing center tutors' activities wdth and perceptions about L2 customers in general. As
the abstract to the dissertation states, "This study began the process of creating a body of
knowledge that looks at the writing center in the writing practices of L2 students"
(Carter-Tod, 1995). David Coogan's doctoral dissertation (1995a), on the other hand,
deals with an emergent and innovative practice among writing centers, i.e., the use of
electronic mail to conduct writing center tutorials. This work discusses the three models
of operation that have characterized writing center pedagogy and three uses of computers
in the field of Composition Studies. This dissertation, which examines six e-mail
tutorials of NESs, however, does not include an analysis of e-mail tutorials intended for
EFL leamers.
A few published articles explore ESL students' expectations and offer suggestions
for writing center tutors on how to deal with ESL students who go for a consultation to
the writing center (e.g., Harris & Silva, 1993; Kennedy, 1993; Powers, 1993; Thonus,
33
1993). Most—if not all—of these studies are based on sporadic observations of what
occurs during ESL students' visits rather than on carefully planned and long-term
observations. Finally, even though quite a large number of projects have been conducted
with the aim of establishing contacts among a variety of students and audiences (see
below), very few of these projects have focused on EFL writing, the revision stage, and
the value of NESs' feedback. Issues like these have been more frequently addressed in
the research conducted in the domain of NES writing instmction.
1.3.2. Computer-mediated Communication in NES and ESL/EFL Settings
Another component or aspect examined before this research project was
conducted is CMC theory and practice, since it is through the use of computers as a
means of communication that this project was implemented and carried out. Networkedcomputer instruction has been used for about fifteen years now with different results.
The philosophical assumptions underlying the incorporation of computers into ESL/EFL
writing instruction have also undergone a transformation from a view of computers as
tutors that can correct errors and reinforce emerging skills and good linguistic habits to a
view emphasizing the concept of computers as tools that facilitate and enhance written
communication between diverse individuals and groups of individuals who are unlikely
to meet or correspond with one another in real-life circumstances.
Research in CMC in ESL/EFL and NES settings has offered mixed conclusions as
to the pedagogical usefulness and benefits inherent to the use of network-based writing
instruction imparted through local and wide area networks (LANs and WANs). This is
34
one of the reasons that the same skepticism or resistance to technology often expressed
by educators and researchers in English composition studies is evident among ESL/EFL
teachers. Nevertheless, and in spite of this resistance, the transition from a
"transmission" to a "functional/interactional" pedagogical model signals evolution rather
than stagnation. In effect, in Teaching ESL writing, Joy M. Reid (1993) stresses that
more and more ESL/EFL instructors are advocating CMC because "networked computer
classrooms currently offer the most promise for enhancing collaborative and studentcentered classroom leaming" (p. 43). In ESL/EFL teaching and leaming, ftirthermore,
the belief in the benefits of immersion methods and in the importance of having real
audiences to speak and write to is implicit in the interaction that takes place through
CMC between NES and ESL/EFL leamers.
1.3.2.1. CMC in NES Settings
The following discussion covers issues pertinent to CMC and has been subdivided
into two subsections, CMC in NES settings and CMC in ESL/EFL settings, for the sake
of clarity and organization and in order to address topics specific to each field. In trying
to establish the pedagogical advantages and disadvantages of using computer-mediated
communication through LANs and WANs in writing classes in the US, composition
speciahsts have conducted a considerable number of studies. These speciahsts have
evaluated the benefits and risks of using computers in writing classes and, in so doing,
they have foregrounded many theoretical issues. Two of the benefits that have been
considered include the increased amount of text production on the part of students placed
35
in computer-networked classrooms, as compared to the amount of text produced by
students placed in regular classrooms, and the peculiar characteristics of this fluid
conversation-like text written on-line (Batson, 1993a, 1993b; Ferrara, Bmnner, &
Whittemore, 1991; Kemp, 1993, 1995; McComb, 1993; Murray, 1988). Another important advantage brought to the forefront by practitioners and researchers is the facilitation
of activities such as invention heuristics and revision of students' drafts through peer- and
teacher-critiques that the use of CMC offers (Kinkead, 1987; Mabrito, 1991; Marx, 1990;
Palmquist, 1993).
Furthermore, and from a more theoretical angle, the important and politically
weighted issues of totalitarian and democratic classroom practices, most clearly
visualized as teacher authority and student empowerment, have been carefully analyzed.
Some of these analyses have tended to highlight the advantages of computers as a
medium that allows for cooperative and functional leaming environments and the
building of communities between individuals who would not otherwise interact (Hartman
et al., 1991; Kinkead, 1987; Palmquist, 1993; Riel, 1983, 1985; Siegel, Dubrovsky,
Kiesler, & McGuire, 1986). Other studies have also emphasized benefits such as
students' empowerment and their sharing of authority with the teacher through the free
expression of their opinions and ideas (Cooper & Selfe, 1990; DiMatteo, 1990; Kremers,
1990; Selfe & Meyer, 1991), the concomitant decrease or limitation of the frequently
reported teachers' interventionist role (Barker & Kemp, 1990; Cooper & Selfe, 1990;
Hawisher & Selfe, 1991), and social status equahzation in decision-making among group
36
members in educational settings (Cooper & Self, 1990; Dubrovsky, Kiesler, & Sethna,
1991; Mabrito, 1992).
As a result of the alleged liberating potentials of the computer, the emergence of
certain discriminatory attitudes, as well as the development of what has been labeled
"flaming" behavior (i.e., the utterance of words and opinions considered inappropriate for
a classroom setting), has been detected by some composition practitioners and
researchers. This is why some solutions recommending that teachers take the role of
moderators to counteract prejudicial comments or actions that intimidate some students in
the computer-networked classroom have been proposed (Gmber, 1995; Romano, 1993;
Takayoshi, 1994; Warshauer, 1995). Some thinkers have seen through the veils of both
computer dangers and blessings, trying to strike a balance between the advantages and
disadvantages that computers may bring to the writing classroom (Bump, 1990;
Hawisher, 1992). The attention paid by CMC researchers in LI environments to subjects
such as the amount and type of text, power and authority, and collaboration and
interaction indicates that there exist important issues that should not be overlooked in any
study involving language leamers in any setting.
1.3.2.2. CMC in ESL/EFL Environments
Some of the insights and theoretical considerations pertinent to CMC in NES
settings mentioned in the preceding paragraphs can apply to ESL/EFL writing instmction.
In fact, issues such as collaboration, interaction, negotiation of mearung, and the building
of communities of leamers have been emphasized as important advantages computers
37
have to offer the foreign language leamer. In an attempt to determine the real benefits of
using computers as a medium of communication, some L2 specialists have explored ways
of establishing contacts between different cultures and reported their findings in
numerous studies (Barson, Frommer & Schwartz, 1993; Chun, 1994; Cummins, 1986;
Cummins & Sayers 1990, 1995; Goodwin, Hamrick, & Stewart, 1993; Healy Beauvois,
1992; Kehn, 1992; Sayers, 1993; Smith, 1990; Soh & Soon, 1991; Warschauer, 1995,
1996). These contacts facilitate—these researchers contend—communication,
collaboration, and interaction with real audiences. More specifically, this electronic
communication supposedly allows for reinforcement of loosely leamed material, for the
students' exposure to new vocabulary and the everyday or colloquial language used by
native speakers of the L2, and, in tum, for the development of language skills in the
target language. Other studies, also dealing with the acquisition of an L2, address
theoretical issues conceming collaborative leaming and suggest how activities conduci\e
to collaborati\e leaming can be implemented through the use of CMC (e.g., Renie &
Chanier, 1995; Warschauer, 1997).
Additional studies have addressed important issues such as the pedagogical successes and failures in computer networking across states and across countries. Some of
these studies have suggested and/or demonstrated how target language skills can be
developed and how cultural awareness can be attained (Cohen & Miyake, 1986; Kem,
1995; Riel & Levin, 1990; Tillyer, 1993). The supposed advantages that the interaction
between EFL leamers and NESs can produce in the process of developing the writing
skill of EFL students abroad, however, still remains to be explored and demonstrated
38
more conclusively. For example, more than one hundred accounts of projects and
activities involving EFL students or other L2 leamers and the use of networked
computers have been featured in Warschauer's Virtual connections (1995). Nevertheless,
these valuable reports constitute anecdotal information about innovative classroom
practices rather than theory-based and carefully implemented research.
A considerable large number of articles that report on the effects of using
computers, certain word-processing software packages, and style and grammar checkers
have been published in some joumals such as CALICO, System, and the electronic
joumal TESL-EJ. A few examples are articles by Chen (1997), Hyland (1993), Lam and
Pennington (1995), and Liou (1991, 1993). The purpose of most of these studies,
however, consists in assessing the advantages and disadvantages of an interaction
between leamer and machine. They do not often address the impact of using computers
as means of communication between human beings.
Two recent studies go beyond the impact of word-processing tools and the use of
computers on the quality and quantity of writing produced by ESL students (Braine,
1997; Sullivan & Pratt, 1996). These two studies compare ESL classes where the
students worked together with their teachers without the mediation of computers (the
traditional classroom environment) and ESL classes where computers were used as tools
for communication between peers and between students and instructor (networked
computer classrooms, in LANs). These studies, however, examine the quality of writing
and students' attitudes toward the use of networked computers in ESL environments in
universities in the US. In fact, most studies conducted so far deal with ESL populations
39
in the US, the UK, Japan, China, Malaysia, and some Arab countries. The present study,
instead, explores a different situation: the attitudes, perceptions, and textual revisions of
EFL leamers, all of them native speakers of Spanish in a South American country, who
received comments about their writing via e-mail from on-line writing center consultants
in the US. Besides the focus on an EFL group of students in a monolingual Spanishspeaking country, the other important difference between most of the studies to date and
this one resides in that this study involved asynchronous communication through a WAN
rather than synchronous communication through a LAN.
An overview of unpublished works that examine the writing processes of
EFL/ESL students and their use of computers reveals a focus on one of the following
issues: the types of revision strategies employed by the students, the kind of feedback
provided by teachers, peers, and grammar/style checkers, or the usefulness of computers
as word processors. Most of these dissertations, however, have studied ESL populations
in the US or the UK. Overall, these unpublished works explore the use of computers (a
particular word-processor and/or grammar/style checker) and networked computers as
factors that may facilitate or hinder the writing process (Brock, 1993; Butler-Pascoe,
1990; Ghaleb, 1993; Hassan, 1995; Kitchin, 1991; Mehdi, 1994; Rahman, 1990;
Singer, 1994; Smith, 1993). There are only a few dissertations incorporating these three
variables simultaneously: L2 writing, the revision process, and computers as means of
communication rather than as word-processing tools (e.g., Herrmann's, 1990; SmithHobson's, 1993). It appears that unpublished works encompassing all these issues and
specifically focusing on EFL populations in monolingual foreign countries are non40
existent. Again, to date it seems safe to state that there is no unpublished work that
brings together the revision process of Spanish-speaking EFL student writers and NES
on-line writing center consultants and that examines the use of computers as mere
"mediators" in the communication between EFL leamers and an audience of NESs.
As discussed above, ahhough the amount of research in the field of ESL/EFL
writing has increased in the last two decades, the research studies to date have not
explored the possible advantages of having writing center consultants participate in the
writing process of EFL leamers. Most of the research in the revision stages of ESL/EFL
leamers has focused on students' revising strategies, students' reactions to peer and
teachers' comments, and the transformation of students' texts upon the reception of both
peer and teacher feedback. Studies focusing on these issues have produced interesting
findings. These results, however, cannot be applied or generalized to all ESL
populations, much less extrapolated to EFL populations in monolingual settings, because
of some problems with research designs or simply because these studies examine small
populations or populations with very specific characteristics. The present study addresses
some of the insights derived from previous research, albeit by investigating a small group
of subjects, but also incorporating a new variable in the revision stage of EFL students:
the participation and collaboration of on-line writing center consultants.
As far as CMC is concemed and as illustrated above, the use of computers in the
teaching of LI and L2 has sparked considerable interest among teachers and researchers.
Many studies that incorporate CMC as a variable have been conducted in the field of
ESL/EFL leaming. Most of these studies, however, have focused on general aspects of
41
L2 leaming such as the possibilities of encouraging cuUural understanding and of
interacting with native speakers of the target language so as to leam vocabulary and
practice some language skills. In many cases, some of these studies speculate on the
benefits of CMC without the bases of a carefully planned research design that would
permit some of their claims. Furthermore, there have not been many clear attempts at
investigating the one skill that—presumably—can be more readily and efficiently studied
when using computers, i.e., the writing skill. Those published and unpublished works
that examine the writing skills of ESL/EFL students and that incorporate the use of
computers as a variable have focused primarily on the role played by the machines and
the software employed when writing and revising rather than on the assistance that other
human beings (peers and/or teachers) can provide during these processes.
Because of the general nature that characterizes current research that incorporates
the variable of CMC in ESL/EFL situations, this study focused on only one aspect of
ESL/EFL leaming, namely, the writing skill. Furthermore, as most of the research about
ESL/EFL writing has revolved around the impact of the use of word-processors and style
and grammar checkers and around the differences between networked-computer
classroom and the traditional oral classroom, both the interaction between ESL/EFL
leamers and NESs and the impact of NESs' suggestions on ESL/EFL writing have not
been flilly explored yet. This study, therefore, was designed to contribute with the
incipient research in this area by studying the reactions and perceptions of a small group
of EFL leamers about the feedback received from the Texas Tech University On-line
Writing Center consultants in the US, the textual revisions these students incorporated in
42
their final drafts, and the nature of these electronic comments sent via e-mail. The focus
on EFL leamers (Spanish speakers) in a foreign country and the incorporation of CMC
and writing center consuhants' feedback as variables differentiate this project from
similar projects studying the revision process.
In this chapter, two main issues have been addressed: the general concepts that
provide the theoretical framework and rationale for this research project and the review
of the literature pertinent to the research questions. The remainder of this dissertation
covers the various aspects of the research conducted in partial fulfillment of my doctoral
work. Chapter 2 discusses all the methodological considerations pertairting to this case
study. Chapters 3 and 4 present the main findings. Chapter 3 addresses the attitudes,
perceptions, and reactions of the subjects as revealed by the interpretation of the surveys
and interviews. Chapter 4 examines the students' textual revisions and the types of
comments from the On-line Writing Center consultants. Finally, Chapter 5,
"Conclusions," presents a summary of the main findings and a discussion of the
limitations of this study, and it also suggests possible routes for future research on the
same topic.
43
CHAPTER 2
METHODOLOGY
As stated in the previous chapter, the purposes of this study are to explore and
understand advanced EFL students' behavior when revising their written work, to identify
the characteristics of the commentary native English speakers working at the Texas Tech
University On-line Writing Center send these students, and also to establish whether
improvement occurs from the first to the final version of two short essays written by
these EFL leamers.
This study falls within the definition of qualitative descriptive research in that it
seeks to explore and understand the issues mentioned above without controlling or
isolating variables, as would occur in the case of quantitative research. As Lauer and
Asher put it in their glossary of terms when referring to this type of research
methodology, "Qualitative researchers typically neither establish treatment or control
groups nor quantify variables" (p. 285). The fact that the word "qualitative," as opposed
to "quantitative," has been used to identify the type of research conducted here does not
necessarily mean that frequency counts and percentages have automatically been
excluded from the report of results. As will be seen in Chapter 4, figures representing
frequencies and percentages have been employed in order to report findings and compare
data. Within the parameters of qualitative descriptive research, the case study
methodology has been used in this research project.
44
In Case study research: Design and methods. Robert K. Yin refers to the use of
multiple sources of evidence as one of the principles of data collection for case studies
(1989, pp. 84-95). As the characteristics of case studies dictate (Yin, 1989; Lauer &
Asher, 1988), a variety of data collection methods have been used for this particular
study. Two surveys or questionnaires, one interview, field notes (researcher's
observations), students' writing samples, and the On-line Writing Center consuhants'
electronic messages constitute the five main sources of information used both to analyze
EFL students' behaviors and the nature of on-line writing center feedback and to report
results.
Among the six sources of evidence that Robert K.Yin (1989) includes in his book
is "participant observation." Yin defines "participant observation" as "a special mode of
observation in which the investigator is not merely a passive observer. Instead, the
investigator may take a variety of roles within a case study situation and may actually
participate in the events being studied" (p. 92). Even though I have not listed this source
of evidence among the methods of data collection enumerated above, it is important to
acknowledge that I, acting simultaneously as researcher and instructor, became a
participant-observer in this study. I did not actually send my work to or receive
feedback from the On-line Writing Center consultants, like the other participants did, but
I played two roles simultaneously, that of the researcher and that of the instructor. In
other words, I acted as a participant in that I was present giving directions, organizing
and supervising all of the activities of the EFL students in their British Literature class.
45
while at the same time I was gathering information: taking down notes, collecting
documents, and administering questiormaires.
It is important to keep this particular situation in mind and weigh its advantages
and disadvantages. Although being the participant-observer entails certain advantages
over the more traditional role of the external detached observer in that it offers a
"distinctive opportunity...to perceive reahty from the viewpoint of someone 'inside' the
case study rather than extemal to it" (Yin, 1989, pp. 92-3), at the same time the role of
participant-observant, or researcher-instructor, involves certain risks associated with
preconceived tendencies or biases. In other words, as a participant observer, I may have
been able to detect aspects or details of this EFL environment that otherwise I would not
have been able to see, but also—and at the same time—I may have assumed the existence
of certain tendencies that may have only existed in my mind, due to my personal
involvement in the project.
In this chapter, information conceming methodological issues will be discussed
and analyzed. These issues will be addressed in four main sections: Sections (2.1) and
(2.2) will cover students' activities and the features of the computer classroom: more
specifically, the characteristics of the setting and participants in the study, the
justification for the selection of subjects, and aspects related to the implementation of the
project. Section (2.3) will provide information about the procedures used for gathering
data and the different sources of data obtained, i.e., participants' short essays, the On-line
Writing Center responses, field notes, surveys, and interviews. Section (2.4) will deal
with data analysis, more specifically with the classification and coding of data. This is
46
the longest of all the sections in this chapter since it examines a number of subtopics
including the elaboration of classification tables, the identification of idea units in
complex on-line writing center comments, and the training of two independent raters who
employed these classification tables.
2.1. Setting and Participants
2.1.1. Research Site
This project was conducted at "Instituto Juan Zorrilla de San Martin," in the city
of Cordoba, Argentina. Unfortunately, there is not a direct equivalent in the US
educational system to this type of educational institution in Argentina that would permit a
comparison between the two. "Instituto Juan Zorrilla de San Martin" is a teacher-training
college that imparts education at the tertiary level and that offers two different degrees:
one degree in teaching English as a foreign language and another one in business
administration. Students can register in this college to begin a four-year study program
leading to either degree only after having obtained their secondary school diploma. The
teaching certification allows graduates from this institution to teach English to EFL
leamers in primary and secondary schools (for the EFL study program at the research
site, see Appendix A).
This teacher-training college is directed and administered by a board of directors.
The academic director, who is also a member of this goveming board of directors,
supervises and coordinates the work of faculty members and fiilfils all types of
academics-related tasks. This educational institute is financially supported by its students
47
and by the state government. After registration and in order to be able to attend classes,
the students pay a monthly tuition/administrative fee of approximately $50, yet some
students can obtain scholarships that help reduce this monthly fee. Besides the support
from its student population, this tertiary institute receives funding from the state
government, which provides a bigger financial contribution than does student tuition in
the form of teachers' salaries. Most of the courses (19 out of 24) are taught exclusively in
English; five courses out of the 24 are taught in Spanish.
A considerable number of students attending this institution could be labeled
"non-traditional" students. Some of these students aheady have a job and a family to
look after. All classes are therefore scheduled in the evening from 6:45 to 11:15 p.m.
Furthermore, the majority of the students enrolled in this institution with the purpose of
becoming teachers of English at the primary and secondary school levels are women who
come from various social backgrounds and may have previously begun and then quit
college education. Moreover, upon starting a study program in this college, students often
evince a wide range of abilities in their command of the English language. In other
words, some of them have only taken English classes in high school, some of them have
taken English classes since they were in kindergarten, others have lived in an Englishspeaking coimtry for a while, some others have taken English courses at the University of
Cordoba, and still some others have already been hired to teach English to young
children. All these differences point to the diversity of the student population. There
exist two leveling elements, though. On the one hand and as mentioned above, all of
students have completed their high school education, since they cannot register for
48
classes if they have not done so. On the other hand, all the students are native speakers of
Spanish. The fact that they have the same native language in common differentiates them
from ESL populations in the USA, for example, since the latter groups usually
encompass people with various language backgrounds.
Another characteristic about the student population in this college is that a great
proportion of the students come to this institution after having failed in some of the
courses they were taking in other institutions such as "Sagrado Corazon" (which is an
educational institution at the tertiary level similar to "Instituto Juan Zorrilla de San
Martin," but with a longer study program and with higher fees) or the School of
Languages (National University of Cordoba), which is reputed to be a very strict and
difficult institution. Consequently, the language proficiency of the students who begin
their studies in this teacher-training institution varies considerably from student to
student. These differences, which are apparent at the beginning of their studies, tend to
smooth out as the students advance in their study program.
This study was conducted during 1997. The academic year for tertiary
institutions and universities in Argentina extends from approximately the beginning of
April to mid November (March and December are frequently final examination months).
The initial number of participants in this study was thirty-one. These participants
included two groups of intermediate and advanced EFL students who were respectively
taking two different literature classes and who were at two different stages in their degree
program. One group was taking British Literature II (and some other classes) in order to
fulfil the requirements for the third year of the study program at this institution. The
49
other group was taking courses corresponding to the last year of studies, among them
British Literature III. As explained earlier, I acted as the participant-observer, playing the
roles of researcher and instructor for both these groups of students. For all the students
participating in the study, this was the first time they had ever sent their compositions
electronically to and received feedback from on-line writing center consuhants in the
USA. In spite of the fact that thirty-one students participated in the study, not all the data
obtained from the thirty-one participants were analyzed. In other words, for this study
there was a purposeful selection of subjects.
2.1.2. Selection of Subjects
Since this study sought to identify, in an exploratory and naturalistic fashion, the
characteristics common to on-line writing center consultants' comments directed to
advanced EFL students' compositions and to study and catalogue students' attitudes and
reactions to this new way of obtaining feedback, the data gathered were used selectively.
In other words, as the aim of this study was not to generalize findings and conclusions to
a larger population, as a researcher I decided to carry out an in-depth description and
analysis of only nine of the thirty-one subjects who participated in the study. As a
researcher, and in order to rationalize and account for this decision, I followed the
justification proposed by Erlandson, Harris, Skipper, and Allen in Doing naturalistic
inquiry: A guide to methods (1993). These authors justify the use of purposive sampling
in naturalistic research, and in so doing they contend the following:
Central to naturalistic research is purposive sampling. Random or representative
sampling is not preferred because the researcher's major concem is not to
generalize the findings of the study to a broad population or universe, but to
50
maximize discovery of the heterogeneous pattems and problems that occur in the
particular context under study. Purposive and directed sampling through human
instrumentation increases the range of data exposed and maximizes the
researcher's ability to identify emerging themes that take adequate account of
contextual conditions and cultural norms. (Erlandson, Harris, Skipper, & Allen,
1993,p.82)
Keeping the above reasons in mind, carefully considering the pros and cons of studying
the whole group of thirty-one students or the small group of nine students, and following
the "purposive sampling" approach (Erlandson, Harris, Skipper, & Allen 1993, pp. 8285) for the selection of subjects, in my role as a researcher I decided to focus on the
nine students taking British Literature III. Authors such as Patton (1990) and Miles and
Huberman (1994) devote considerable attention to sampling strategies for quahtative data
in naturahstic research. These two writers provide useful taxonomies for the selection of
subjects according to a particular purpose or strategy (Patton, 1990, pp. 169-183; Miles &
Huberman, 1994, p. 28). For this study, a combination of sampling strategies derived
from these taxonomies was applied to choose the subjects for in-depth analysis of data.
The group of subjects chosen constituted "critical cases" and conformed to a "criterion."
The most important sampling strategy used was the one that justifies the selection
of "critical cases," and the parameters that guided the selection of subjects followed
Patton's definition of "critical cases." According to Patton,
Critical cases are those that can make a point quite dramatically or are, for some
reason, particularly important in the scheme of things. A clue to the existence of
a critical case is a statement to the effect that "if it happens there, it will happen
anywhere," or vice versa, "if it doesn't happen there, it won't happen anywhere."
The focus of the data gathering in this instance is on understanding what is
'° In his recent book. Qualitative Inquiry and Research Design, John. W. Cresswell calls this
approach for the selection of subjects "purposeful sampling strategy" (1998, pp. 118-120).
51
happening in the critical case. Another clue to the existence of a critical case is a
key informant observation to the effect that "if that group is having problems,
then we can be sure all the groups are having problems." (Patton, 1990, p. 174)
For this study, the group of students taking British Literature III was selected for a
detailed analysis and triangulation of data because they constituted what can be
considered critical cases: all of them were taking the last classes before graduation, and
therefore, they supposedly possessed the highest or most advanced English language
skills at the research site. This situation made them critical cases because, being at the
last stage of their study program and possessing an advanced level of language ability,
these students were in a better position to understand, judge, and eventually use feedback
in order to improve their compositions. The other group of students, which was not
considered for analysis, still had to take more classes than the selected group before
graduation. This situation indicates that the group not chosen for analysis had a lower
level of proficiency in the target language and therefore may have had to face more
difficuhies when both interpreting and incorporating feedback.
Two other characteristics that make these nine subjects critical cases include the
type of reading and writing they had to do. On the one hand, the literary works the
students in the chosen group-those taking British Literature III—were required to read
were easier than those the other group of students were assigned to read. The literary
works students taking British Literature III had to read had been written by 19th and 20th
century authors, while students taking British Literature II were supposed to read literary
works from the 17th and 18th centuries. Obviously, reading 17th and 18th century
literary works would impose more difficulties on any person-whether he/she be a NES
52
or NNES-than reading works from the 19th and early 20th centuries, due to the fact that
gaps and differences between the English used nowadays and the English used in those
earlier works are bigger and more significant than those between present-day English and
the Enghsh used in the 19th and 20th centuries.
On the other hand, the nine students taking British Literature III were chosen in
favor of those students taking British Literature II because the former were supposed to
write longer pieces than the latter for their written assignments. Those students taking
British Literattire III were required to write short essays rather than paragraphs. It was
assumed that these longer pieces—which involve more topic development and therefore
more use of language—would allow me, as researcher, to find, observe, and examine
more features about both on-line writing center comments and students' follow-up
changes than would the shorter pieces of writing produced by the students taking British
Literature II.
These three characteristics, i.e. the higher language proficiency of the British
Literature III group coupled with the type of reading material they had to work with and
the writing required of them, make them a critical group since, as Patton argues, "if that
group is having problems, then we can be sure all the groups are having problems"
(Patton, 1990, p. 174). Furthermore, and expanding on Patton's statement, I would add
that if the group is benefiting from this experience, then we can be sure other groups can
be, or actually are, doing so too.
In addition to matching Patton's description of critical cases (1990), the group of
students selected for in-depth study also conformed to two related criteria. Patton argues
53
that "the logic of criterion sampling is to review and study all cases that meet some
predetermined criterion of importance," and he later adds, "the point of criterion sampling
is to be sure to imderstand cases that are likely to be information-rich because they may
reveal major system weaknesses that become targets of opportunity for program or
system improvement" (Patton, 1990, pp. 176-77). The two criteria met by the students in
this group were the particular class they were all taking and the type of writing they were
required to do in that class. As mentioned above, all these students were taking British
Literature III and were also required to write longer pieces of writing (short essays) than
the students in the other group, who were taking British Literature II. The former were
required to write a minimum of three paragraphs per essay: an introductory paragraph, a
development or body paragraph, and a conclusion, between approximately 300 and 400
words. The latter, however, were required to write only a well-developed paragraph,
between approximately 150 and 200 words. Furthermore, the students in the chosen
group had taken almost all the courses that comprise the program of study at the
institution they were attending. This particular situation made them more experienced
and mature students, thus also more valid informants. In other words, the group of
students who became the focus of analysis constituted "information-rich cases for study
in depth" (Patton 1990, p. 169), since they were in a position to provide more insights
into the research questions guiding this exploratory study. Having chosen one group as
the focus for in-depth analysis does not mean, however, that the data produced from the
other group were discarded. The in-depth analysis of the other twenty-two students'
compositions is part of a forthcoming article that compares the two groups of students.
54
2.2. Implementation: Students' Tasks
and the Computer Classroom
One basic component of both British Literature courses involved the reading and
class discussion of literary works included in the syllabi. The testing of the students'
knowledge of the theme, characterization, setting, and style of the literary works assigned
for reading was carried out by having the students write paragraphs and/or short essays
about these topics. It was these documents written by the students that provided part of
the raw data for analysis. The subjects of this study were supposed to write their
compositions in longhand the day they had a test.'^ As instructor and researcher, I
collected these hand-writtdn documents and brought them back the following class so that
the students could type them up. After their essays had been typed and saved on a
computer disk, they were sent through e-mail to the Texas Tech University On-line
Writing Center. Finally, after receiving the corrmients from the On-line Writing Center
consultants, I printed them out, took them to the research site and distributed them to the
corresponding students. These documents containing the comments from the On-line
Writing Center provided another set of raw data for analysis.
Not all the participants in this study were familiar with the use of computers;
consequently, some measures were taken in order to minimize the novelty and
apprehension that using computers for the first time could entail. In order to prepare the
students for these exchanges, which required the use of computers and which took place
" As written assignments were going to be graded and as most of the students were
somewhat apprehensive about or not familiar with the use of computers, it was decided that, in
order to ease their anxiety about computers, the students would write their first drafts as they
were used to doing, in longhand. This issue is explained in more detail later in the discussion.
55
at the end of April, mid September, and early November 1997, the students were allowed
to go to the computer classroom for some informal practice during one class period (for a
brief description of activities, see pp. 58-59 below). Using the computers to type a few
sentences helped the students familiarize themselves with some basic computer and
keyboard functions. After this hands-on practice in the computer classroom, the actual
implementation of the project followed three main steps:
1. One class period, approximately one hour and a half, was devoted to the students'
writing a composition in longhand. In writing these short essays, the students responded
to one prompt of their choice. The prompts or questions for the written assignment were
intended to test the students' knowledge of the literary work under consideration. This
activity took place in a regular classroom (see Appendix B).
2. A second class was devoted to the students' transcribing these compositions by typing
and saving them onto the computer server in the computer classroom. These short essays
would later be e-mailed to the On-line Writing Center at Texas Tech University.
3. A third class consisted in the students' receiving a print-out with the comments from
the On-line Writing Center consultants and rewriting their short essays in longhand in
class, after having read and considered this feedback. This third and final draft would
receive a grade.
As researcher and instructor, I worked on the assumption that these students had
deah with the ideas and tenets of writing as process with other teachers, more specifically
with their language teachers. That is to say, these students were acquainted with the
different stages in the writing process such as idea generation, drafting, and writing.
56
Even though they were not used to a particular terminology, the students handled the
concepts involved in what Flower and Hayes call "planning," "translating," and
"reviewing" (1981, p. 373). As the three steps described above show, the students were
allowed to engage in a process that somehow comprised these three writing stages. The
students were aware that the instructor would not grade their first drafts. They also knew
that only after receiving feedback from the On-line Writing Center and after having had
the chance to revise their work would they submit their final drafts for a grade.
Many researchers have pointed out that the novelty of using computers can act as
a motivational force that may prompt students to write. Other researchers, however, have
wamed us about anxiety problems on the part of some students who are faced with a
machine that they do not know how to operate (Pennington, 1996, p. 26). That is why,
in order to avoid the degree of fear and anxiety that the use of computers may produce in
new users, the subjects in this study had some practice before actually typing the
compositions to be e-mailed to the US.
As indicated above, the students first wrote their short essays in longhand. The
transcription (i.e., typing) of their handwritten compositions or first drafts was reserved
for the following class. There were a number of reasons why the students wrote their
short essays in longhand first. This written assigrmient would be eventually evaluated;
that is, the students would receive a grade for it, so most of the students who were not
familiar with computers and/or typing would probably feel uneasy or anxious about
having to incorporate new elements such as computers and typing in an exam situation.
So, in order to ease their anxiety or dispel any fears related to computer phobia, as an
57
instructor I decided-and the students agreed-that they would wnte their assignments as
they were used to, i.e., in longhand. Another reason why I had the students write their
compositions in longhand first was to avoid one of the dangers pointed out by Williams
(1992, p. 15); namely, that students composing directly on a keyboard may tend to
conflate oral and written style. Furthermore, keeping in mind Gerrard's early warnings
about the facts that students may be distracted from more important issues such as
content and organization when they have to use sophisticated formatting procedures and
that they may also find it very difficult to write in a computer classroom or computer lab
buzzing with activity (Gerrard, 1989, pp. 104-6), I had the students vmte their first essays
in a regular classroom.
A few students knew how to operate a computer because they either had a
computer at home or were supposed to use one in their jobs, but the majority of students
did not know how to use a computer or how to type. Aware of this situation, I decided to
devote one class period to provide the students with the basic information they needed to
be able to use the computer keyboard and save their work so that they would be ready to
do the same when they actually typed their compositions. Explaining how a computer
works and the basic keyboard functions was not an easy task, simply because students
were going at their own pace. The different pace at which the students were working and
their individual concems required the presence of the instmctor with two or three
students sometimes simultaneously. Some students, who already knew the basics of how
to operate a computer, however, helped the others in the process. After this hands-on
practice class, the students had not leamed everything about word-processors or about
58
how to use them, but at least they had become acquainted with the basics of computer use
and were mentally prepared for the task they were going to engage themselves in on the
day they would type their handwritten first drafts. Once again, this hands-on practice
prior to the actual typing of the compositions that were to be sent electronically was
meant to dispel the commonplace and frequently reported fears and anxiety that the use
of computers often produces in first-time users.
Once all the compositions had been typed by the students and these electronic
documents had been saved one by one onto a computer disk, in my role as the researcher
I e-mailed every one of them to the Texas Tech University On-line Writing Center. I had
to copy and paste every document from the individual files onto Eudora 1.5.2 (the
interface used to send e-mail messages), and only then was I able to e-mail them. As can
be inferred from the preceding information, the exchange of drafts from students to Texas
Tech University On-line Writing Center and back from the writing center to the students
took several days, mainly because the school did not have access to the Intemet. The fact
that an intermediary person was necessary for the whole process to occur meant a slowdown in a process which could have taken less time. The time lapse between the day the
students wrote their first draft and the day they got a response was sometimes more than a
week. It is important to keep these particular circumstances in mind since the passing of
time may produce positive but also distractive effects in the students. The fact that there
was a rather long waiting period from the day the students wrote their first drafts to the
day they wrote the final drafts may have caused them to lose interest in their assignment,
to forget the aim of the assignment, or-what may be even more problematic to them-to
59
forget the information that they had been reading and that was, therefore, fresher in their
minds on the day the initial draft was composed than on the day the final version was
written.
2.3. Data Collection Procedures
For the data collection in this study, a qualitative research methodology was
employed. This qualitative methodology, however, was supplemented by the use of
some quantitative techruques such as frequency counts and percentages, especially in the
analysis of the survey data. The four methods of data collection employed—each of
which yielded the corresponding set of data—include written documents (participants'
compositions and writing center responses), field notes, surveys, and interviews. In order
to enhance the validity of this study, and as recommended by many research methods
specialists (Marshall & Rossman, 1995; Patton, 1990; Yin, 1989), these four types of data
and data collection methods were triangulated. In other words, in order to determine
trends and account for discrepancies apparent in the various sources of information, data
were meticulously studied and carefully compared (see Chapters 3 and 4). The following
sections explain how these various data were collected and coded.
2.3.1. Participants' Short Essays
The students taking British Literature III had to perform a number of tasks and
were evaluated through their performance in an oral presentation and by means of several
written tests. Only few of these written tests included multiple-choice or true-false
60
questions, but all of them contained a short-essay writing section. All of these shortessay writing activities were open-book tests; that is, the students were allowed to have
and consult the literary work they were writing about as well as an English-English
dictionary.
Furthermore, the students were given the chance to think about and revise
their work. On the one hand, the fact that students had to transcribe or type their first
drafts onto a computer in the following class so that their compositions could be saved on
computer disks and then be eventually e-mailed to the Texas Tech University On-line
Writing Center provided them with an opportunity to rethink and modify, if necessary,
what they had originally written. On the other hand, the students wrote their
compositions for a third time after having received the On-line Writing Center
consultants' feedback. The dates the students performed the various tasks are specified in
Table 2.1 below, and the time elapsed between the different activities for each exchange
is displayed in Table 2.2 below:
Table 2.1. Dates the students performed the various tasks during the school year.
(For the topics and general written assignment requirements, see Appendix B)
Writing first draft
Typing draft
Receiving feedback and
writing final draft
April 17
April 22
May 13
September 2
September 4
September 25
October 30
November 4
November 18
'^ It is a common pedagogical practice among EFL instmctors to require that their
students—especially the more advanced students—use monolingual dictionaries like the ones used
by NESs. In fact, in teacher-training colleges, bilingual dictionaries like the ones translators or
interpreters often use are discouraged on most circumstances and absolutely not allowed in exam
situations.
61
Table 2.2. Time elapsed between the different activities for each exchange.
Time elapsed between
Time elapsed between e-
writing and typing the first
mailing the first draft and
draft
receiving a response
First Exchange
5 days
21 days
Second Exchange
2 days
21 days
Third Exchange
5 days
14 days
All of the students' typed first drafts corresponding to the three exchanges were
kept on electronic files identified by each writer's name. The final drafts, which had
been written by the students in longhand, were later typed by the researcher to facilitate
the work two independent raters would eventually do when classifying the effects of the
students' revisions. The last exchange, however, has not been included for data analysis
because this group of essays received two sets of feedback or commentary: the
instructor's and the On-line Writing Center consuhants' feedback. As this was the final
chance the students had to raise their grades before the end of the school year, in my role
as an instructor, I considered it reasonable and fair to provide them with extra help by
commenting on their drafts. Because of the double feedback the students received on
their first drafts during this last exchange, it would be difficuh, if not impossible, to
discem final draft changes made following the instructor's comments from those made
following the On-line Writing Center consuhants' feedback. Consequently, this double
feedback contaminated the last set of data, rendering them unusable.
62
2.3.2. On-line Writing Center Responses
Whenever the short compositions were sent to the On-line Writing Center at
Texas Tech University, the writing consultants had been informed in advance about the
nature of the assignment and the approximate number of compositions they would be
receiving. Once the On-line Writing Center's answers with feedback about the students'
compositions had arrived,^^ printouts were made. These hard copies of the On-line
Writing Center responses were taken to the research site so that each student could read
the comments and revise his/her composition in class. The final product, after the
students made the necessary corrections and/or revisions, was to receive a grade by the
instructor-researcher. As not all the on-line responses arrived together on a single day, as
an instructor I thought it would be more advisable to wait until I had gathered all the
comments so that I could take them to school and have all the students work on their final
drafts during the same class period.
There were a number of difficulties associated with the reception of the On-line
Writing Center responses. Once the responses started arriving, I had to save each
response onto a diskette so that a record of all the interactions could be kept for later
reference and use. Furthermore, as mentioned above, copies or printouts of all the
responses had to be made so that they could be distributed to the corresponding students,
who would need them in order to introduce changes in their written assignments. This
long drawn-out process was demanding and time-consuming. If the school had had an
" The students' essays were e-mailed from and the Texas Tech University On-line
Writing Center responses received at a computer terminal at ComputEnglish Center, a privately
owned school in the city of Cordoba, where EFL and the use of computers are taught.
63
Intemet connection, the researcher-instructor would not have had to act as an
intermediary in the process of sending and receiving compositions. In other words, the
students could have sent their own compositions and received the corresponding
feedback individually and thus obtained feedback faster.
Another difficulty was that some of the Writing Center consultants omitted
writing the full name of the students to whom they were sending their response. As a
consequence, on a few occasions 1 was forced to decipher who the addressee was by
carefully looking at the students' drafts and at the consultants' comments so as to match
the essay with its corresponding response. An additional problem involved the difference
in the kind of feedback provided by the different consultants. In other words, most
consultants sent very complete and detailed responses, while a few of them just sent short
comments without much information to help or guide the students. This is the nature of
writing centers, however. Writing centers benefit from the work of many consultants,
who obviously have their own personality and way of responding to papers. These
differences are not to be criticized or avoided, but should be taken into account when
considering the effect the responses had on the students' final drafts.
Some other minor problems had to do with the students' interpretation of the
messages from the On-line Writing Center. They understood almost everything, yet
when they did not understand some comment, they resorted to the instructor for help.
This happened very infrequently, though. Two of the nine students approached me and
asked me for clarification of one comment during the first exchange, and about three
students did the same during the second exchange. These questions, however, did not so
64
much reflect lack of understanding on the part of the students; the students' questions
instead reflected the need for reassurance about their comprehension of the consultants'
commentary and reassurance that the changes they were about to incorporate into their
essays met the expectations and approval of their instructor/grader.
Before the students actually started working on their revisions, I wamed them
about and explained to them the presence of some typographical characters in unusual
places in some of the On-line Writing Center consultants' responses. Due to the
incompatibility between the sofhvare used in Argentina for sending and receiving e-mail
messages (Eudora 1.5.2) and one of the systems used by the consultants in the US (VMS
system), capital letters such as R and S, for example, would appear at the beginrung and
end of a phrase or sentence indicating the beginning and end of quotes, respectively. In
other words, these two letters appeared in the place of quotation marks; e.g., "manners"
would appear written as RmannersS in some of the responses the students received. The
capital letters U and T could appear in place of an apostrophe; e.g., "John's" would appear
written like "JohnUs" or "JohnTs" in some of the On-line Writing Center responses.
Furthermore, two carets ( » ) at the beginning of each line meant that the designated lines
belonged to the recipients' own text. In some cases, some On-line Writing Center
consuhants would use special characters to indicate that they were inserting a comment
within the student's text. A sequence of asterisks (*****) or square brackets ([[[ ]]]) at
the beginning and end of a paragraph meant that the students were reading a consuhant's
comments, depending on the consultant's responding techniques. Even though these
unusual typographical characters did not appear to prevent or hinder understanding, they
65
had to be pointed out so that the students would not be distracted or confused by their
presence in the consultants' text.
In approximately 60% to 65% of the cases, the On-line Writing Center responses
contained long and complex comments. These comments included idea units that could
be broken up into different comments. Therefore, and for the purpose of the forthcoming
data analysis, which would entail the description and classification of comments, an
operational definition that would help separate one comment or idea unit from others was
developed. The task of breaking up the long and complex comments into individual
comments was performed both by the researcher and by an extemal reader. The
complete set of on-line responses was given to an independent reader so that she could
separate the comments on her own, using the operational definition she had been
provided with (for a more detailed explanation, see "Operational Definition of
Comment," on pp. 83-86 below).
2.3.3. Field Notes
The field notes consist of all the descriptions and narratives that, as participant
observer, I wrote in the form of diary entries after most of the classes I taught in the
research site and particularly after each exchange between the EFL students and the Online Writing Center had taken place. I typed my observations and reflections about
students' actions and reactions to the project and kept a record of all this information in
various electronic files, to which I have resorted time and time again after the study was
carried out, as a way of bringing back fresh memories of what I saw happening while the
66
project was underway. These field notes consist of approximately twenty pages of
double-spaced typed information. The relevance and value of these notes reside in that
they provide snapshot pictures of what occurred during the process of data collection.
Most of these written notes, however, also reveal problems encountered while conducting
the project. The problems pointed out in these notes serve as caveats that may be
instructive not only for the planning of future research with similar populations in similar
settings, but also for the development of classroom practices that involve the use of
computers and the application of these innovative strategies.
The two most outstanding findings revealed by these notes are the enthusiasm
shown by the majority of the students who participated in this research project and some
technical problems that can and should be taken care of if a similar research project is
ever conducted in the same site (see Appendix J, for a sample of these notes). Upon the
researcher-instructor's announcing the project and describing the procedures through
which it would be implemented, the two groups of students showed interest in
participating in the project: all of them agreed to participate in the study by signing an
acceptance statement (see Appendix K), and they also seemed to appreciate the chance
they would have to write a few drafts and receive feedback from English instructors
abroad before their writing assignments were assigned a final grade.
Apart from the apparent benefits or advantages this project brought about for the
students, it also presented certain difficulties. Two of the most critical problems as
revealed in the field notes relate to the students' lack of experience with the technology
available, and also-and more importantly-the limitations of the technology the students
67
and I as instructor and researcher had access to. Even though there was a general feeling
of enthusiasm among the participants about the prospective responses from the On-line
Writing Center consultants in the USA, in my roles as an instructor and researcher, I also
noticed some concem on the students' part while they were typing their compositions on
a computer keyboard. The concem seemed to derive from their lack of computer
training, since most of the students—though being able to type—did not know some
keyboard functions such as "caps lock," "shift," "tab," "backspace," or "delete," and did
not know how to pull up a word processor, open a file, or save their documents. As
indicated above, to alleviate these problems, however, I had the students practice using
the computers two classes prior to their typing the short essays in answer to the test
prompts. Unfortunately, this hands-on practice had not been enough since some students
still experienced some difficulty when trying to operate the computers the day they typed
their essays.
Among the technical obstacles that I had to surmount as a researcher was the
computer classroom itself. The computer laboratory where the participants typed their
short essays was equipped with twenty-one computers linked together through a server.
The network system in use at the time of this project was Novel 3.12. In spite of the fact
that most of the computers were networked and they all had Windows 3.1 installed, the
students did not have access to a single word-processor. Most of computers were
equipped with the same word processor (i.e., Microsoft Works for Windows), but some
of them had a different one (Microsoft Word 5.0 or Word Perfect 5.0). This difference
produced some difficulty when I had to explain some word-processing functions to the
68
students, especially because students had to be given different directions according to the
word processor they had access to on the computers they were using.
One serious problem, indirectly related to the equipment available in the computer
classroom, was the limited possibility for communication between the computer
laboratory assistant and me as an instmctor-researcher. This limitation did not result
from unwillingness to communicate on either part. Schedule conflicts and the fact that
the computer lab assistant did not have a telephone hindered a fluid communication. The
computer specialist and I were able to talk about the server's and computers' operation
approximately three times during the whole academic year. Notwithstanding these
limitations, I managed to use the lab without major problems, but only by investing long
hours in trying to understand and deduce the essentials of operating the server on my
own.
In addition to these various inconveniences, the most limiting one, as has been
noted above, was that the research site did not have access to the Intemet, and without
this access, the students did not have access to electronic mail to send their written
assignments to the Texas Tech University On-line Writing Center. Even though the
institution's academic authorities and board of directors had approved of this project,
their approval to obtain the services of a commercial Intemet service provider remained
on their waiting list for the rest of the academic year. On seeing that the students' work
was not going to be sent from the computer classroom at the institute, in my role as a
researcher I decided to save all the students' files and then send them to the Texas Tech
69
University On-line Writing Center from ComputEnglish Center, a private institute that a
colleague and I used to own and mn in the city of Cordoba.
2.3.4. Surveys
The participants in this study answered two different surveys or questionnaires
especially prepared for this study. These questionnaires were designed to identify the
participants' perceptions about their writing behavior in the target language before and
after participating in the study. It should be noted that, in deciding to have the subjects
answer questionnaires, I, as a researcher, was aware of the drawbacks that often
accompany survey answers. One of these drawbacks derives from reliance on the selfreports from the surveyed participants. As Marshall and Rossman explain.
In deciding to survey the group of people chosen for study, researchers make one
critical assumption—[that the characteristics, beliefs, or attitudes of a population]
can be described or measured accurately, through self-report. In using
questionnaires, researchers rely totally on the honesty and accuracy of
participants' responses. Although this limits the usefulness of questionnaires in
delving into tacit beliefs and deeply held values, there are still many occasions
when surveying the group under study can be useful. (1995, pp. 95-6)
In spite of these problems, which may result in lack of validity, as a researcher I
considered that the administration of questionnaires would provide one more source of
data. The information obtained from the surveys would prove useful for the purposes of
triangulation of data. In other words, this information would eventually help to
corroborate, strengthen, or even question conclusions drawn from other sources of data.
Another advantage of using surveys or questionnaires is that "substantial amounts of
information can be collected in a relatively short time" (Brown, 1988, p. 3). In fact, the
70
questionnaires used for this study, which contain between twenty to forty questions, were
answered by the participants in twenty to thirty minutes.
The first survey was administered at the beginning of the academic year, before
the students sent their compositions to the Texas Tech University On-line Writing Center,
and the second one was administered by the end of the academic year, once the students
had had the opportunity to send their compositions to the On-line Writing Center at least
twice. The participants did not take the surveys home to answer the questions there. On
the contrary, both questionnaires were administered during the first portion of a regular
class period so as to avoid a low response rate, a danger commonly reported in the
hterature (Brown, 1988, pp. 3, 185; Lauer & Asher, 1988, pp. 67-8). Having the
participants answer the surveys in class was a good decision, since some of the
respondents had some doubts and questions that were clarified for the whole group as
they were voiced by the students.
These questionnaires did not require that the respondents supply their name, yet
in order to facilitate the task of triangulation during data analysis, each questionnaire was
numbered and distributed one by one to each student while attendance was being
checked. All the participants provided their names at the end of the surveys. This
identification of subjects became useful later, during the processes of data analysis and
reporting of findings.
The first survey was designed to obtain information about the students'
perceptions of their proficiency level in English and their attitudes and reactions to
writing in the target language (see Appendix L). This first survey was divided into three
71
sections. The first section dealt with the students' knowledge of the English language,
the second one revolved around the students' attitudes and activities while re-reading a
composition first draft with the purpose of revising it, and the third section inquired about
the students' activities when actually incorporating changes to their drafts during the
revision process. The second survey was designed to obtain information about the
students' attitudes, perceptions, and reactions to the type of feedback they had received
from the On-line Writing Center consultants (see Appendix M). This second
questionnaire was also divided into three different sections: the first section focused on
the students' opinions about the implementation of this project and on their reactions
conceming the comments received; the second section was meant to uncover the students'
perceptions about the nature and usefulness of the feedback received from the On-line
Writing Center consultants; the third section inquired about the students' views on the
project as a whole.
Both surveys included multiple-choice and yes-no questions. The multiple-choice
questions incorporated, in most of the cases, only four options for the respondent to
choose from (e.g., "always, frequently, occasionally, never"). There were only few
instances of questions that offered more than four options as possible answers. These
surveys did not include open-ended questions since these would be included in the
interviews to be conducted by the end of the school year. There was no pilot-testing of
the surveys with a reduced group of people, as is customarily the case (Marshall &
Rossman, 1995, p. 96) and emphatically recommended by specialists (Lauer & Asher
1988, p. 65). However, the surveys were read and revised several times by the researcher
72
and one of her colleagues. Furthermore, the research advisor provided insightful
comments and suggestions for fine-tuning questions, for changing wording and/or
phrasing that could cause confusion to the respondents, and for dividing the surveys into
different sections. The nominal data obtained from these surveys were entered into
Microsoft Excell 97 so that frequency counts could be run and percentages calculated.
2.3.5. Interviews
Many specialists in qualitative research methodology refer to various ways of
conducting interviews (Cresswell, 1998; Erlandson, Harris, Skipper, & Allen, 1993;
Marshall & Rossman, 1995; Patton, 1990). Patton (1990), for example, explains that
"there are three basic approaches to collecting qualitative data through open-ended
interviews," and he classifies them like this: "(1) the informal conversational interview,
(2) the general interview guide approach, and (3) the standardized open-ended interview"
(p. 280). The interview I designed for the subjects in this study falls into the "interview
guide" category, which consists of questions that address a set number of issues but that
need not be asked in a certain order. In other words, the interviewer is relatively free in
that he/she can develop a sort of "conversation" with the interviewee rather than follow a
strict pattem of questions. As Patton explains.
An interview guide is prepared in order to make sure that basically the same
information is obtained from a number of people by covering the same material.
The interview guide provides topics or subject areas within which the interviewer
is free to explore, probe, and ask questions that will elucidate and illuminate that
particular subject. Thus the interviewer remains free to build a conversation
within a particular subject area, to word questions spontaneously, and to establish
a conversational style-but with the focus on a particular subject that has been
predetermined, (p. 283)
73
The interviews for this study were conducted with Patton's remarks and advice in mind
(1990). As a researcher, I intended the interviews to follow a dialogue mode or
conversational format rather than to follow a rigid question-answer pattem. I believe this
is in fact how the interviewees saw the interview situation too, as a way of exchanging
ideas and opinions rather than as a process of merely answering questions.
In addition to offering general suggestions for the tasks of designing and
conducting interviews, qualitative research specialists have provided useful information
and wamed prospective researchers against possible problems during the interviewing
process that may diminish the quality of the data obtained and may obstruct the analysis
and interpretation of such data (Marshall & Rossman, 1995; Patton, 1990; Yin, 1989).
Patton (1990), for example, devotes one whole chapter to discussing common and serious
problems that can be taken care of by means of a careful preparation on the part of the
interviewer. Patton offers useful recommendations conceming the types of questions that
should be asked and the particular wording or phrasing that should be used when
formulating the questions so that the interviewer actually obtains the information that
he/she needs to obtain. As far as the wording of questions, Patton stresses that good
questions "for purposes of qualitative inquiry .. . should, at a minimum, be open-ended,
neutral, singular, and clear" (1990, p. 295).
Even though most of the questions in the interview complied with the guidelines
provided by Patton (1990), there were some questions that did not fit exactly into Patton's
definition of good questions because they presented a dichotomy to the interviewee (e.g..
Question 1. "Have you ever had a person read your composition before you tumed it in
74
for a grade?"), or they were not really singular questions (e.g., Question 6. "What is more
important for you, help in grammar or help in the organization and content? I mean,
when you are revising a composition that you wrote, if you have time to go over the draft,
what do you often pay more attention to? to the grammar and formal aspect or to the
content and organization?"). However, these questions-like all other questions-were
written and included in the interview guide to fulfill the role of prompts. In fact, during
the course of the actual interviews the phrasing of the questions changed, yet the key
issues being probed remained the same (Appendix N). Most—if not all—of the
interviewees were ready to talk and provide their opinions. In spite of the fact that they
had to answer "yes-no" questions, the interviewees would often elaborate on their
answers, by giving more details, rather than provide a simple "yes" or "no" answer.
Patton also wams researchers against the use of "why" questions on several
grounds. On the one hand, he argues that "'Why?' questions move beyond what has
happened, what one has experienced, how one feels, and what one opines, and what one
knows to the making of analytical and deductive inferences" (p. 313). In other words,
"why?" questions are often difficult to answer because interviewees are forced to make
causal inferences. On the other hand, Patton (1990) explains that there could be a myriad
of answers to a "why?" question because there usually exist several levels of reasons or
explanations—such as economic, philosophical, social, and personal reasons-depending
on the question asked. Therefore, it may be easier for the interviewee to prioritize a
reason in his/her answer and leave all the other reasons aside. The problem becomes
apparent when the researcher begins to analyze the data and he/she discovers that the
75
answers to the same question from different interviewees refer to different levels. This
situation, as Patton states, often leads to data that are simply unusable. Finally, "why?"
questions may be tricky because they "can imply that a person's response is somehow
inappropriate" (Patton, p. 315). In other words, the interviewer must be careful not to
give the impression that he/she questions the validity of an interviewee's opinion or point
of view. All of the above are reasons that justify my avoidance of "why?" questions
during the interviews.
Eight of the nine students in this case study were interviewed once they had sent
two of their short essays and received some feedback from the On-line Writing Center.
Only one of the nine students who had participated in the project was not interviewed.
Participation in the interviews was optional. As an instructor and researcher, I did not
want to exert more pressure by imposing an extracurricular task on these students who, in
most cases, in addition to their schoolwork, had their families to attend to and had job
schedules to comply with. Therefore, I decided to interview those people who
voluntarily agreed to be interviewed for about twenty to thirty minutes, before the
regularly scheduled classes. The interviews were spread out during a period of
approximately three weeks. As it tumed out, eight out of the nine students taking British
Literature III were interviewed. As the interview was mainly intended to corroborate or
clarify information obtained from the informants' survey answers, I did not think it would
be imperative to interview every single student. As long as I interviewed more than half
the participants, I would probably obtain a good picture of how reliable the data from
surveys were.
76
All of the interviews were taped and transcribed. The transcription process,
however, did not take place immediately after the actual interviews had taken place, as is
often recommended (Patton, 1990). The end of the school year activities and the lack of
funds to pay for a free lance or independent transcriber forced the researcher to postpone
the transcription until a few months later, once the academic year was over. The
transcription consisted in typing the interviews by means of a word-processing program
and keeping each interview in a different file.
2.4. Data Analysis: Classification and Coding of Data
One of the main purposes of this study involved the description of the type of
comments the Texas Tech University On-line Writing Center sent advanced EFL leamers
in an EFL teacher-training college in Argentina. Furthermore and more importantly, the
study also aimed at determining the positive or negative influence these comments had on
the writing of these students. The following sections will cover various aspects of the
classification and coding of data. First, a description of the two taxonomies employed for
the classification of on-line writing center comments and students' revisions will be
provided. Second, the task of breaking up the long comments into smaller idea units by
an independent reader and the researcher will explained. Third, an account of how the
training sessions were conducted will be given. These three sub-sections, especially the
last one, are intended to explain and illustrate how the data obtained from the On-line
Writing Center consultants and those resulting from the comparison between first and
77
final versions (after the On-line Writing Center feedback had been read and used by the
students) were assessed, classified, and coded.
2.4.1. Taxonomies for Data Analysis
In order to fulfil the first objective of this study, i.e., to explore the nature of online writing center comments for revision, I applied a taxonomy especially designed for
this study. This taxonomy is not totally original in that it incorporates concepts and
categories featured in an existing classification table for teachers' comments, but it is new
in that it derives from the application of a comparative method of observation and
analysis of content data. The original research model created by Dana Ferris (1997) for
her study "The influence of teacher commentary on student revision" (see Appendix C)
provided a springboard for the new taxonomy that was designed for this study.
Upon
pilot-testing Ferris' model, I found out that there were a number of issues such as the
length of the comments, the use of hedges, and the differences between imperative,
affirmative, and interrogative statements, which had been considered by Ferris but which
did not constitute relevant information to this study. Therefore, I decided to omit these
items from the revised classification table. Moreover, I observed that there were some
specific issues such as the explanation of the nature of a problem, the request for a
change, and the direct supply of words or phrases in the electronic responses from the
'•* Ferris' original taxonomy is described in more detail in D. Ferris, S. Pezone, C. Tade, and
S. Tmti's article "Teacher commentary on student writing: Descriptions and implications," published
in 1997 in the Joumal of Second Language Writing, 6, 155-182.
78
On-line Writing Center consuhants that were not accounted for in Ferris' original
taxonomy because they were not relevant to her study.
After the careful study of the On-line Writing Center consuhants' commentary,
therefore, the following categories were incorporated into the new taxonomy, due to their
apparent relevance: explanation of the nature of the problem (category 2); explanation of
the nature of the problem and suggestion for solution (category 3); explanation of the
nature of the problem and supply of the correct words, phrases, and/or sentences
(category 4); direct suggestion for solution (category 5); and direct supply of words,
phrases, and/or sentences (category 6) (see Taxonomy A, in Appendix I). These
categories somehow collapsed or lumped together the eight categories proposed by Ferris
and did away with differences between the interrogative, imperative, and statement forms
that this researcher had used in her taxonomy.
In order to develop a taxonomy that, besides incorporating concepts already tested
by other investigators, would also capture the nature of new and probably different data,
as a researcher I applied "the constant comparative method" (Glaser and Strauss, 1967).
This method is used in research that aims at producing theory rather than testing it.
Glaser and Strauss's method is based on the researcher's careful and meticulous
examination of the data so as to create a taxonomy or model that reflects the particular
characteristics of the data under study. In order to build a classification table that would
capture what I, as a researcher, considered salient and representative characteristics of the
commentary provided by this particular group of On-line Writing Center consultants and
that would consequently be appropriate for this study and group of subjects, I followed
79
the inductive approach proposed by Glaser and Strauss (1967). Their approach
"combines, by an analytic procedure of constant comparison, the explicit coding
procedure of the [hypothesis testing] approach and the style of the theory development of
tiie [hypotiieses generating approach]" (Glaser and Strauss, 1967, p. 102).
This method of designing classification tables derived from qualitative or content
data has been used in studies about teacher and peer feedback by researchers such as
Dana Ferris (1997) and Lockhart and Ng (1995). Lockhart and Ng explain, for example,
that "one important concept in this approach is to allow categories to emerge from the
data, rather than imposing [sic] preconceived categories on the data" (1995, p. 614). In
addition to looking for general categories that allowed me to group certain comment
characteristics under a certain label, I compared each category with all the comments
coded under the same label to ensure a certain degree of similarity among the various
instances of similar comments. As Lockhart and Ng point out, "This comparison allows
the researcher to refine the categories and to identify properties associated with the
category" (1995, p. 614).
The new taxonomy consisted of twelve main categories grouped in two sections,
which I will call taxonomies A and B. The six categories displayed in the first section, or
taxonomy A (see section A, or taxonomy A, in Appendix D, for on-line writing center
comments classified according to pragmatic intent), are related to and seek to establish
the differences between the types of consultants' responses to texts (i.e., actual reactions
to and questions about the students' texts) and corrective feedback or language models
often supphed by instructors (Fathman & Whalley, 1990; Kepner, 1991; Brandl, 1995).
80
The second section of the new taxonomy's final design, or Taxonomy B (see section B, or
taxonomy B, in Appendix D, for on-line writing center comments classified according to
substance), includes five language categories previously examined in a number of studies
dealing with the analysis of teachers' comments, and the additional category "nonapplicable." In other words, these five categories-content, organization, vocabulary,
grammar, and mechanics—were also incorporated as five different categories in this
taxonomy for comments not only because of their traditional validity, but also because
they were readily identifiable while pilot-testing the On-line Writing Center consultants'
commentary. These more traditional categories have been the focus of many studies
about foreign and second language writing (for example, Cohen & Cavalcanti, 1990;
Hedgcock & Lefkowitz, 1996) and also constitute the bases for long-standing writing
evaluation procedures (Jacobs, Zingraf, Wormuth, Hartfield, & Hughey, 1981).
However, in spite of the relevance of these five categories to the research questions, as a
researcher I needed to obtain information that would not merely conform to more
traditional or conventional classification tables, but that would also provide insights into
the nature and characteristics of a teaching resource that is barely beginning to be
explored. That is why the other six categories featured in Taxonomy A for on-line
writing center comments, referred to above, were incorporated and used in this research
project.
In order to fulfil the second objective and answer the second research question;
that is, the kind of effect on-line writing center comments have on advanced EFL
students' final drafts, I employed another taxonomy. In this case, the taxonomy derives
81
directly from tiie taxonomy designed by Dana Ferris (1997) for her study about the effect
of teacher feedback on students' revisions. The new taxonomy applied in this study
includes most of the concepts captured by Ferris' original taxonomy but constitutes an
abridged version of the original. In addition, this new taxonomy incorporates a category
absent from Ferris' analytic model (see Appendix E for the original and new taxonomies).
The reason behind this abridgement resides in the fact that after reading and
assessing the changes from first drafts to final versions made by the nine participants in
the first and second exchanges, as a researcher I envisioned the difficulty of providing
operational definitions that would clearly express the difference between these two
concepts in Ferris' model: "mirumal attempt by the student to address the comment" and
"substantive change(s) made by student in response to comment" (Ferris, 1997, p. 322).
As a researcher, I decided to integrate these two concepts under only one category (i.e.,
"change made by the student in response to comment") so as to make the classification
easier, while realizing, however, that important differences would be unexplored and lost
in the simplification. Nevertheless, this decision is justified by the exploratory nature of
this study. The added category corresponds to the label "avoidance of problem." The
reason I decided to include this category springs from an initial analysis of the students'
final versions. Upon classifying changes from first drafts to final versions, I noticed that
in some instances, the student had omitted or deleted words, phrases, or sentences that the
On-line Writing Center consultant had pointed out as problematic. Even though one can
only speculate about the reason behind a student's decision not to make a recommended
change, it was clear that in some instances the students had deleted the problem word or
82
phrase and, in so doing, avoided tackling the issue at hand. This occurrence was
infrequent, but as a researcher I believed these few instances were worthwhile to point
out and study. That is why this category was incorporated into the new model.
2.4.2. Operational Definition of Comment
It is important to note that in order to simplify the task of the two independent
raters who would later apply the taxonomies for types of comments, I as a researcher, in
agreement with an independent reader, separated the On-line Writing Center comments
by indicating their begirming and end. Once the comments had been set off by the use of
square brackets and nimibers for easy identification, the raters would only have to
concentrate on classifying each comment on a grid, by using the corresponding
taxonomy. After this comment classification, the raters would proceed to indicate the
effect the comment produced on the students' final drafts. In order to attain a certain
degree of stability or consistency (a type of reliability, according to Krippendorf, 1980,
pp. 130-1) when separating the different idea units that make up long comments, a second
reader marked the beginning and end of each comment using the same operational
definition I had previously used to identify comments.
The operational definition used was the following: "A comment is made up of any
word, phrase, sentence, or combination thereof, that provides information, suggests or
requests a change, asks for explanation or clarification, gives an example, and/or supplies
the correct form for a specific part of the student's text and about a single topic." Before
the second reader applied the definition on her own to indicate the begiiming and end of
83
each comment, I explained how I had applied the definition by using three responses to
three different students as examples. After the explanation and examples, the second
reader applied the same definition to three other responses. After the reader had
separated the comments one by one, the reader and I discussed doubts and points of
discrepancy. Later on, the reader worked on her own to divide or separate, into idea
units, the long comments in the eighteen On-line Writing Center responses.
Evidently, the application of the definition of "comment" by two readers did not
yield identical results (see Appendix F). Even though the rehability coefficient was not
calculated, the researcher's markings and those of the second reader agreed in 97.08% of
the cases for the first group of responses (those belonging to the first exchange) and in
100% of the cases for the second group of responses (those corresponding to the second
exchange). The percentage of agreement for the two groups of responses (i.e., the
responses corresponding to the first and second exchanges combined) was 98.54%. It is
important to note that even though this last percentage was considerably high, this result
does not constitute a true reflection of complete agreement. This percentage was
calculated using the total number of comments, yet the global number does not capture
the individual differences per paper.
The cases where there was lack of agreement were discussed so as to provide
valid reasons that would justify the compromise or agreement. For example, I considered
this complex comment: "Overall, I think you make a clear point. Make sure you look
through the essay for the **'^, since I pointed out many sentence-level problems" just one
final comment, but the independent reader considered there were two different comments
84
in this passage. In this case, as a researcher I decided to follow the independent reader's
division because it clearly reflected more closely the operational definition of comment
being used. In other words, in this comment two different issues are clearly addressed:
the fact that the student had made a clear point and the fact that the consultant wanted to
make stire the student would go over the sentence level problems indicated. It was
therefore agreed that this passage consisted of two comments. (This example was
extracted from paper #2, first exchange).
In the following example, I identified two comments, while the reader identified
only one. However, the reader had noted on the margins: "Each sentence here talks
about a different thing. So I was not sure." Upon discussion, researcher and reader
agreed to divide the passage in question into two parts: (1) the first part dealing with the
need to supply evidence through the use of quotes, and (2) the second one requesting
elaboration or explanation of some ideas. The two comments finally agreed upon have
been set off by numbers and the carat symbols (< >) in the example: (1) <"Provide
evidence of this by quoting from the text. Use quotations from the text to support your
claim that Pip's pride never allows him to admit this.> (2) <Then go on to explain why
Pip's pride never allows him to admit this. Does he not admit or show he appreciates
Mrs. Gargery's efforts to raise him because he is afraid doing so will indicate weakness?
You need to explain Pip's failure to admit or show his appreciation for Mrs. Gargery.">
(This example was transcribed from paper #3, first exchange). (For more examples, see
Appendix G). All the differences and discrepancies that became apparent upon
comparison of the extemal reader's with my markings for comments were discussed, and
85
an agreement was reached before the On-line Writing Center responses were distributed
to the two independent raters who would classify the comments according to their
pragmatic intent and to their substance or content.
2.4.3. Training Sessions
Before the training of the two independent raters took place, I pilot-tested the two
taxonomies. As a researcher, I conducted the pilot-testing procedures in order to identify
possible problems and thus to be better prepared to train the extemal raters. The
taxonomy for on-line writing center consultants' commentary that comprises two sections
(one related to the apparent intent of the comment and the other one related to five
traditional language sub-areas) and the taxonomy for the effects of students' changes were
pilot-tested with two groups of data. The taxonomies were applied to (1) data that were
unusable and therefore would not be later analyzed (six first drafts and the corresponding
On-line Writing Center responses and final versions belonging to students who had been
excluded from the study) and (2) the pool of data to be later analyzed by the two
independent raters (the eighteen first drafts. On-line Writing Center responses, and final
versions belonging to students who had been selected for the study). As a result, twentyfour sets of grids with the corresponding classification categories per comment were
produced (see Appendix H).
After the data had been coded by the researcher, the training of the independent
raters took place. During the first training session, the taxonomies used contained scanty
information (see Appendices D and E) compared to the more detailed information
86
handed out to the raters during the second training session (see Appendix I). The first
training session consisted in going over three sets of commentaries from the On-line
Writing Center consultants at Texas Tech University and three students' first drafts and
their respective final versions. As far as the comments are concemed, I had previously
set the comments off by using square brackets, had numbered them, and then classified
each comment following the classification criteria mentioned above. First, the raters and
I together went over three examples that I had already classified, analyzing every instance
of doubt and/or discrepancy and reaching an agreement on how to classify problematic
comments. Second, the raters classified the three remaining on-line commentaries,
keeping the already classified responses as a guide. Finally, the raters' classifications of
these three new responses were checked against each other. In order to train the two
independent raters in the use of the criteria for classifying the effects of changes made by
the students in their final drafts, I proceeded in a similar fashion. In this case, however,
there was only time to examine and discuss the examples that I had already classified. In
other words, the raters did not have the opportunity to classify the effects of students'
revisions on their own.
Even though the labels or categories for the different comments are
straightforward in theory, problems arose when trying to classify comments such as the
following ones:
(1) "It might help to mention a specific act or speech by Satan that shows he is proud.
This would offer proof for your claim that Satan suffers from pride." (comment #7,
sample #2)
87
(2)". . . second, the in [sic] '...the powers God has given him...' does this 'him' refer to
Satan or Adam?" (comment #17, sample #2)
(3) "Doesn't Satan see Paradise, become envious, almost regret his fall, and then take on
the snake's body? Be careful that ideas and events from the literature are being connected
in the correct order." (comment #19, sample #2)
(4) " . . . what's your opinion? Do you agree that Satan and humanity share common
traits / weaknesses such as pride, wrath, and envy? Are you convinced? If so what in
Milton's poem convinced you? If not, why hasn't Mihon convinced you?" (comment
#21, sample #2)
The difficulty in assigning comments such as these to one of the classification
categories lies in the fact that it is not easy to discem whether a problem has been pointed
out or identified by the consultant and a solution also been offered (category #3 in section
A of the taxonomy for comments; see Appendix I), or whether a solution has been
offered without an indication of the nature of the problem (category #5 in section A of the
taxonomy for comments; see Appendix I). It was agreed, however, that comments such
as #7 and #19 above (examples 1 and 3) should be classified as "indication of problem +
suggestion for solution" (category #3) because even though the problem is not explicitly
identified or labeled by the consultant, the second part of the comment hints at the
problem and thus helps the student see where the draft needs more work. It was also
agreed that comments similar to #17 and #21 above (examples 2 and 4) should be
classified as "suggestion for solution" (category #5) because they contain prompts or
88
questions that clearly request or indicate a course of action, and this suggestion overrides
any implicit identification of the problem.
Another example of disagreement involved a comment about a comma splice. I
had classified this comment as a "mechanics" problem, yet one of the raters explained
that this problem should probably be labeled a "grammar/style problem" on the grounds
that a comma sphce involves more than the insertion of a simple punctuation mark. This
particular instance was resolved by deciding that all comma splice problems (as well as
fused or run-on sentences) would be assigned to the "grammar/style" category.
Furthermore, it was extremely difficult to decide whether certain comments referred to
the organization or the content of the essay. The following three examples display this
problem:
(1) "This is a strong beginning and a good thesis." (comment #2, sample #2)
(2) "Because there is another paragraph yet to come about envy, maybe this paragraph
should only describe "pride".) (comment #6, sample #2)
(3) "This is another strong paragraph." (comment #13, sample #2)
Because of the difficulty in establishing the boundaries between the concepts of content
and organization presented by the above comments, it was agreed that unless it was very
clear to the raters whether the comment under consideration referred to one or the other
category, the comment should be classified as both "content/organization." I explained to
them that if there were too many cases like these, the two categories would eventually
have to be combined under a label like "global issues."
89
Finally, most of the positive comments tend to be general; that is, they do not
refer to any specific language area, as the following examples indicate:
(1) "This is another sti-ong paragraph." (comment #13 in sample #2)
(2) "Good," (comment #20 in sample #2)
(3) "I hope I can work with you again sometime" (comment #23 in sample #2)
In view of this lack of explicit information about the issue being addressed in the above
positive comments, it was also decided that unless positive comments referred to a
particular and identifiable language area like "content/organization," "grammar/style," or
"vocabulary/language use," for example, they would be marked as "N/A" in section B of
the classification table for types of comments. In addition to the classification problems
illustrated above, all other instances of differences were discussed and an agreement was
reached by fine-timing the already existing operational definition for the problem.
Due to the many discrepancies and doubts surrounding a considerable number of
comments and students' changes, a second training session was held. During this second
training session, the raters went over operational definitions and examples for each
category first. Only after they had discussed every category and fine-tuned two of them
(categories #3 and #5; see Appendix I) did the two raters analyze and classify an On-line
Writing Center consuhant's set of responses that contained twenty-three comments
covering almost all the classification categories. They also went over the corresponding
student's first and final versions. The raters worked individually and then compared their
results against mine. There was not a complete agreement between the trainer and the two
raters, or between the two raters themselves. Those cases of disagreement, however,
90
were discussed so that an agreement or a compromise was reached. The main differences
between the two training sessions reside in that more time was spent on the operational
definition of each category and on its corresponding examples during the second training
session, and that, probably as a consequence, agreement between the raters occurred
more frequently during this second meeting than during the first one.
In this chapter, I have covered methodological issues by describing the
characteristics of the research site and the participants. I have also explained the process
of data collection and the classification methods employed to analyze such data. In spite
of some problems associated with the limitations of the research setting and with the
seemingly small population chosen for this study, the substantial number of relevant data
systematically collected and analyzed offers enlightening information that supports
previous research and that could also be applied in the classroom situation and in the
implementation of similar studies. The resuhs obtained from the examination and
comparison of the various sources of data will be discussed and analyzed in Chapters 3
and 4 below.
91
CHAPTER 3
RESULTS: EFL STUDENTS' ATTITUDES AND
PERCEPTIONS AS REVEALED IN THE SURVEYS,
INTERVIEWS AND FIELD NOTES
In this and the following chapter, the results of this exploratory study are
presented and analyzed. As stated earlier, this study sought to observe and characterize
the nature of responses that the Texas Tech University On-line Writing Center
consultants would provide EFL students in a teacher-training college in Argentina.
Another aim of the study was to establish tendencies in the students' attitudes and
perceptions conceming this innovative practice. Furthermore, this study also examined
these EFL students' behaviors through the analysis of textual changes or revisions
incorporated in the students' written work, in light of commentary received from this Online Writing Center. It should be borne in mind that the main tendencies observed after
the data analysis apply only to this group of participants, i.e., the nine EFL students
taking British Literature III and the five On-line Writing Center consultants who
responded to this group of students. These tendencies, however, lend support to alreadyexisting research and, at the same time, raise some questions that call for fiirther
investigation. The most important findings of this exploratory research relate to the
nature of the comments that the Texas Tech University On-line Writing Center
consultants e-mailed the nine students in Argentina and to the apparent repercussions that
the different types of comments had on the students' final drafts. Even though the
92
conclusions drawn from the data analysis cannot be extended to other populations due the
limited size of the population studied, they can still shed some light on issues that have
not been exhaustively investigated yet and can also serve as the foundations or guideposts
for further research in this area of knowledge.
The resuhs of this study will be presented in the following order: First, the
students' perceptions about the revision process, in general, and about their participation
in this project, in particular, will be analyzed and discussed in sections 3.1 and 3.2 of this
chapter, respectively; second, frequency counts and percentages representing the various
types of comments and the different types of changes observed in the students' writing
will be reported, compared and analyzed in sections 4.1, 4.2, and 4.3 of Chapter 4.
On the one hand, the results examined in this chapter derive mostly from
information obtained from the students' written questionnaires or surveys and from
interview data. This set of data also includes, on some occasions, information obtained
from field notes. Although these three sources of information will be presented and
discussed more or less simultaneously, in such a way that they feed into one another, the
first and second surveys will fiinction as the major guideposts around which the
discussion will develop. The results discussed in Chapter 4, on the other hand, derive
from the analysis of two main sets of data: one corresponds to the different types of Online Writing Center commentary and the number of responses received by the students
during the first and second exchanges, and the other one concems the type of changes
introduced by the students in response to those comments also for both the first and
second exchanges. After these data are reported, some tendencies are discussed in terms
93
of frequency counts and percentages. Furthermore, an attempt has made to establish
some tentative relationships between the types of comments received by the students and
the subsequent changes incorporated by the students in their final drafts.
3.1. Students' Perceptions and Experience with
Revision Practices (First Survev)
Most of the information about the students' attitudes and responses to receiving
electronic feedback from trained writing center consultants-all of them native English
speakers—derives mostly from the students' surveys, interviews, and the teacherresearcher's field notes. The most relevant points discussed in this analysis of the data
obtained from the aforementioned sources include the students' experience with and
positive attitudes toward revision strategies, their preference for teachers' rather than
classmates' feedback, and their priorities conceming various aspects of writing (i.e.,
content, organization, grammar, and mechanics) when revising their written work.
From the answers to the questions in section B of the first survey^ ^ (see Appendix
L), it is clear that the students had some familiarity with the concepts and activities
involved in revising a draft according to suggestions from readers other than themselves.
Even though probably not on a regular basis, all the participants in this study had
previously used the help and comments of classmates and teachers. These EFL students
'^ Even though section A of thefirstsurvey provides interesting information about the
students' language background and their perceptions about their language skills, it has not been
discussed in this work because the answers to the questions in this section of the survey provide
insights that do not directiy contribute to answering any of three research questions that guided
this study.
94
considered that, overall, it was "useful" to incorporate suggestions coming from both
teachers and classmates. The majority of the students, eight out of the nine students
surveyed, indicated that they either "always" or "sometimes" reread their written work in
order to check content, organization, grammar, and/or mechanics (questions 1 through 4
in the First Survey, section B; see Appendix L). Furthermore, the majority of the
students (seven out of nine) agreed that it was worthwhile to make changes to the first
draft of a paragraph, letter, composition, or essay that they wrote so as to produce a more
understandable, effective, and pohshed version. Only two of the nine students surveyed
indicated that this activity was not worth the effort.
This positive attitude towards revising a first draft and incorporating changes in
the final version of essays is reinforced by the information gathered from the students'
interviews. The eight students interviewed affirmed that rereading and rewriting their
compositions is worth the effort. Although one of the interviewees admitted to disliking
the process, all of them argued that revising is useful because this process can help
writers see aspects of their compositions that are often overlooked or neglected when the
first draft is composed. They also explained that the revision process is beneficial
because it helps them make changes that eventually contribute to the improvement of
their essays. It should be noted, however, that the two students who in the first survey
answered that it was not worthwhile introducing changes, admitted later, while being
interviewed, that reading and revising a first draft was useful because the writer is able to
see things that he/she did not see before.
95
This contradiction can be explained in terms of the pressure that may have been
imposed on some of the students by the fact that their teacher, in the role of a researcher,
was conducting the interview. The absence of pressure while answering the survey
questions—since the students did so anonymously-may have allowed the students to
express more freely their true feelings about the revision process. This lack of extemal
pressure could perhaps explain why these two students showed a negative attitude toward
revision when answering the survey questions. The pressure of voicing their opinions in
the presence of their teacher—though also a researcher—during the interview may have
driven these students to answer in a positive fashion. It could also be argued that as the
survey was answered at the begiiming of the school year and the interviews were
conducted towards the end of the academic year, time and maturation may have caused
the students to change their opinion. After having received and benefited from feedback,
these two students may have modified their opinion about revision so that they were
ready to admit the positive side of revising their written work at the time the interview
was conducted.
Furthermore, even though the survey and interview questions were related, they
did not inquire about exactly the same issue. While the survey question reads: "Do you
think it is worthwhile to make changes after you have written your first draft of a
paragraph, letter, composition, or essay?", the interview question was not phrased in
precisely those terms in all the interviews. Due to the conversational nature of the
interview, the question was phrased differently in each interview session, for example:
(1) "As far as writing a composition, do you think that rewriting a composition,
96
something that you wrote two days ago, can help you improve that composition?"; (2) "Is
it worthwhile, useful, to write a composition you wrote yesterday, for example? You say:
'Oh , I'm going to look at it again, before giving it to the teacher and rewrite it?'"; (3) "Is
it really beneficial to rewrite a composition after you have written it?"; and other similar
questions. This discrepancy between the answers obtained in the surveys and in the
interviews cannot be attributed, with any degree of certainty, to any of the three
explanations above. Nevertheless, situations of discrepancy hke this one highlight the
importance of the notion of accurately phrasing survey and interview questions so that
they address the targeted issue. As Patton (1990, pp. 295-298) and other research
metiiodologists have pointed out (Erlandson, 1993, pp. 85-91; Yin, 1989, pp. 82-85), the
importance of the careful and accurate phrasing of questions in quahtative research
carmot be emphasized enough.
Section B of the first survey also sought to determine the students' experience
with revision practices. Some of the questions, for example, inquired whether the
students had ever received suggestions from classmates or teachers, and if so, how often
they had used those suggestions. When asked about the frequency with which classmates
had read their rough drafts, both as a voluntary action or as part of an assignment
requirement, most of the students surveyed affirmed that they had often received
feedback from classmates. In effect, out of the nine students surveys, five and six
students, respectively, chose "always" in response to questions B.6 ("Has a classmate
ever read your written assignments before you tumed them in because you asked him/her
to do so as a kind of favor?") and B.7 ("Has a classmate ever read your written
97
assignments before you tumed them in because he/she was required to do so as part of a
class assignment?"). In addition, two and three students, respectively, responded
"sometimes" to the same questions. Only two students responded that they "seldom"
resorted to their classmates' help in order to obtain an opinion about their first drafts.
Only three of the nine students, however, reported that they had actually incorporated
changes suggested by their classmates.
When asked similar questions regarding help or comments provided by teachers
(NNESs), eight of the nine students reported that they had received and incorporated
suggestions from their teachers. Only one of the nine students indicated that she had not
received teachers' feedback in the past. Questions 13, 14, and 15 in section B of the first
survey (Appendix L) inquired about help received from instmctors who were NESs.
Two students did not circle any of the four possible optional answers to question 13
("Has an English teacher—native speaker—ever read your written assignments before you
tumed them in?"), but wrote these two comments in the margins: "only once" and "I've
never had an English teacher (a native speaker)." These comments were used to classify
their answers as "seldom" and "never," respectively. Therefore, the final count indicates
that only one student responded "sometimes," five students "seldom," and three students
"never" to the question of whether a NES had read their written assignments. It is not
very clear under what circumstances these six students who answered "sometimes" or
"seldom" to question #13 received feedback from native-English-speaking instructors.
As the information in the field notes reveals, however, two or three of these students had
corresponded with NESs prior to this project. It is possible that the students who
98
answered affirmatively may have misinterpreted the question. Many of the students may
have corresponded with NESs, but receiving comments geared to their improvement as
writers appears to be a whole different issue. As a matter of fact, according to the
students' description, the type of feedback obtained on those occasions—when they had
allegedly received comments from NESs-seems to have been related to vocabulary or
idiomatic expressions rather than to have included comments pertaining to ways of
enhancing their writing skills or becoming better writers.
If we compare the students' answers about the possible usefulness of classmates'
and teachers' (both NNESs' and NESs') comments, it becomes evident that these students
tend to value teachers' responses and suggestions over those of their classmates. Even
though none of the students stated that classmates' assistance was "not useful at all," all
of them considered a NNES teacher's suggestions "extremely useful" (five students) and
"very usefiil" (four students). When asked question B.21 ("How useful do you think it is
to have an English teacher—native speaker—read your written assigimients and make
suggestions for improvement before you submit them to be graded?"), eight students
indicated that the type of help that a NES teacher could provide was "extremely useful"
(four students) and "very useful" (four students). The same student who had clarified in a
marginal note in her survey that she had never had a NES as teacher did not answer the
question about the possible useflilness of a NES's suggestions for improvement. She
probably left the question unanswered because she did not want to venture an answer
without actually having had the experience of receiving feedback from a NES. We need
to remember that at the time these students answered this survey, they had not yet sent
99
their compositions to nor received responses from the Texas Tech University On-line
Writing Center. In other words, some of the information gathered from this first survey
conceming the students' views about NESs' feedback may have been based on what the
students speculated it would be like to receive responses from NESs directed at helping
them improve their writing.
Although not all the students interviewed were asked this particular question
about their preference for the provenance of feedback or suggestions, a few students
made interesting comments that shed some light on this issue. One student explained, for
example, that if given the opportunity she would prefer to have a teacher rather than a
classmate read and comment on her writing. Two other students mentioned that they had
informally asked other classmates (not as part of a requirement for a class or an
assignment) for their opinion on a few occasions; for example, during breaks between
classes. According to the students' examples, the assistance obtained from these
classmates was often hmited and consisted, in most cases, of simple suggestions such as
"add a comma here and there" or just "change this word." These findings, though not
generalizable to all advanced EFL leamers-due to the small size of the population
studied-corroborate some findings reported in other studies indicating that students tend
to tmst their teachers' more readily than they trust their classmates' comments.
Studies dealing with a comparison of preferences between teachers' and students'
feedback have shown that students tend to follow suggestions coming from their teachers
rather than from their peers (Connor & Asenavage, 1994; Zhang, 1995). One of the
reasons for this preference could be related to the idea that EFL leamers tend to trust the
100
authority of those people who supposedly know more than they do. Another survey-type
study focusing on ESL students' opinions about peer reviews reveals, for example, that
most of the students who participated in the survey (about 55%)) found peer review
sessions beneficial in that these reviews made them "more aware of the needs and
expectations of their audience, helping them to meet the demands of the writing
classroom which their peers [were] reflecting to them" (Mangelsdorf, 1992, p. 278). It
should be borne in mind, however, that 30% of the students in Mangelsdorf s study
showed mixed comments, and approximately 15%) expressed negative opinions toward
peer reviews. Mangelsdorf explains that the students who did not find this activity
productive or beneficial "did not think that they, or their peers, could be good critics" and
also points out that "the largest number of complaints deah with the students' lack of trust
in their peers' responses to their texts" (1992, p. 280). Mangelsdorf's explanation for the
negative opinions voiced by the students in her study could also account for the
preference of the students in this study for their teachers' feedback over that of their
peers.
Related to the issues of reading a first draft with the purpose of checking it for
problems and incorporating changes in subsequent drafts is the issue of priorities during
the revision stage. These advanced EFL students' behavior and practices during the
revision stage are explored by the questions featured in the last section of the first survey.
These questions (questions 5 through 12, in section C of the First Survey; see Appendix
L) focus on four main aspects of writing: the content, the organization of ideas, grammar,
and mechanics. Nevertheless, it has been difficuh to determine, from the analysis of the
101
students' responses, whether the students tend to pay more attention to one aspect of
writing over another one. One of the reasons for this difficulty could be associated with
the phrasing and repetitive quality of the survey questions themselves. Furthermore, all
conclusions derived from the analysis of the students' survey answers should remain
"tentative" due to problems of lack of validity that may surface when one attempts to
make generalizations about the behavior of students based solely on their own
perceptions.
Despite these difficulties however, some frequencies became apparent and should,
consequently, be reported and discussed at this point. A frequency count of the answers
indicates that the content and ideas expressed in a piece of vmting seem to be these EFL
students' secondary concems when compared to the organization of ideas. As can be
gathered from the students' answers to question C. 5. in the first survey ("When you have
the opportunity of writing second drafts, do you concentrate on improving the content
rather than the overall organization of ideas?"), the content of an essay seems to be less
frequently addressed or modified when the students go through the revision process. In
fact, six out of the eight students who answered this question said that they
"occasionally" paid more attention to the content of their writing than to the organization
of ideas.
When compared to the importance attached to grammar and mechanics, overall
organization of ideas seems to take precedence over formal issues. Upon being asked
question CIO ("When you have the opportunity of writing second drafts, do you
concentrate on improving the overall organization of ideas rather than the grammar and
102
mechanics-spelling and punctuation?"), six students indicated that they were more
frequently concemed with organization. Two and four of these six students, respectively,
indicated that they "always" and "frequently" focused on organization rather than on
grammar issues when revising their written work. However, the answers to another
question that probes the same issue but is phrased differently reveal slightly different
behaviors on the part of the students surveyed. When answering question C. 12, for
example ("When you have the opportunity of writing second drafts, do you concentrate
on improving the grammar and mechanics [spelling and punctuation] rather than the
overall organization of ideas?"), four students indicated that they often focused on
grammar and mechanics rather than on organization, and the other four students
expressed that they paid attention to overall organization rather than to grammar and
mechanics.'^ This seemingly contradictory information could be accounted for by what
many research specialists have pointed out about the questionable validity of data
obtained from self-reports (Lauer & Asher, 1988, pp. 65-66). In other words, it is not
always possible to obtain a true reflection of what survey respondents actually do or
think, and therefore conclusions drawn from survey data should be tentative, especially in
cases like this one in which such a small population sample was studied.
Conceming the frequency with which students attend to grammar issues rather
than mechanics, their survey answers to question C.7 ("When you have the opportunity
of writing second drafts, do you concentrate on improving your grammar rather than
'^ I have reported only the opinion of eight students because one of the nine did not
answer this question.
103
mechanics [spelling and punctuation]?") reveal that these students tend to favor revising
for grammatical over mechanical errors. In effect, six students expressed that they either
"always" (three of the six students) or "frequently" (three of the six students) did so.
Consequently, it seems that the students who participated in this study tend to focus on
grammatical problem rather than on mechanics when revising their written work. This
information, however, does not permit making generalizations since in the answers to a
follow-up question (question C.8: "When you have the opportunity of writing second
drafts, do you concentrate on improving mechanics [spelling and punctuation] rather than
the grammar?"), five of the nine students surveyed responded that they "frequently" pay
more attention to mechanics, while four indicated they did so "occasionally."
From some of the survey answers, it also appears that concems related to
grammatical structure and mechanics take priority over content when students revise their
writing. When asked question C.9 ("When you have the opportunity of writing second
drafts, do you concentrate on improving the content rather than the grammar and
mechanics?"), only three of the nine students answered that they "always" (one student)
or "frequently" (two students) focused on content rather than grammar and mechanics.
The other six students indicated that they "occasionally" paid more attention to content
rather than to grammar and mechanics. A follow-up question, C. 11. ("When you have
the opportunity of writing second drafts, do you concentrate on improving the grammar
and mechanics [spelling and punctuation] rather than the content?"), appears to support
the tendency revealed in the answers to the previous question, since six of the nine
students indicated that they focus on grammar and mechanics more often than they do on
104
the content of a piece of writing. Only three of the students expressed that they
"occasionally" focused on grammar and mechanics rather than on the content of their
compositions when revising them. This tendency could be a reflection of teaching
methodologies the students are used to, as some researchers indicate (Cohen &
Cavalcanti, 1990; Hedgcock & Leflcowitz, 1994; Porte, 1996).
From the frequency counts of answers to the questions in the first survey and from
the discussion above, it seems that this group of students tends to pay attention to
grammar and organization over content and also that grammar and organization seem to
be equally important in the students' priorities when revising. Interestingly enough,
however, when answering question C.5 ("When you have the opportunity of writing
second drafts, do you concentrate on improving the content rather than the overall
organization of ideas?"), one of the students wrote in her survey, "I try to improve both."
The same student wrote: "I try to pay attention to everything," in answer to question C.6
("When you have the opportunity of writing second drafts, do you concentrate on
improving the overall organization of ideas rather than the content?"). These two
comments point to the fact it may be difficult, or even imnatural, for some students to
dissect aspects of language and writing in the way they were required to do while
answering these survey questions. Closely related aspects such as "ideas" and "their
arrangement" involve what could be labeled as "global" or "macrolevel" issues, since
these two elements of writing can often and more readily be taken as one rather than as
two distinct categories.
105
Information obtained from the students' interviews has helped to supplement,
though in limited ways, the information about priorities during the revision process
obtained from the surveys. In other words, all the information combined does not seem
consistent enough to suggest clearly identifiable tendencies. The varied and sometimes
contradictory evidence reveals that the students were not able to determine which aspect
of writing they tend to focus on when revising their work. The small size of the
population studied, furthermore, does not allow us to make many generalizations either.
Unfortimately, conclusions caimot be easily drawn from the interview data since
the students seemed to experience difficulty in pin-pointing which aspect of writing they
often prioritize when revising. All of the interviewed students indirectly suggested that
there are many variables that can come into play when revising a piece of writing. Two
out of the eight students interviewed expressed that they paid attention to everything
because it is sometimes difficult to separate one aspect of language from the others. One
of these two students explained that, without being aware of it, she may sometimes pay
more attention to one aspect over another, even though she often aims at taking care of
everything. Furthermore, the same student indicated that sometimes grammar mistakes
are easier to take care of than problems related to global issues such as ideas and their
organization. In addition, four of the participants interviewed pointed out that their
major concem when revising revolves around the content and orgaitization of the piece of
writing. One of these four interviewees explained that her major concem during this
project related to the content and organization due to the particular nature of the class for
which the writing assignments were required, i.e., literature. She probably meant that if
106
the class for which she wrote the short essays had been a language or grammar class, she
would have focused on grammar rather than on the global aspects of her compositions.
With regard to the attention paid to grammar, two students admitted focusing their
improvement efforts on sentence structure problems. One of them explained that when
she writes, she is often confident of her knowledge of the topic or content, so she tends to
concentrate on structural problems rather than on the ideas she wants to express. The
students' inability or difficulty to establish which aspect of writing they often concentrate
on when trying to improve their writing was also voiced by a few students when talking
informally about these issues. This difficulty in determining which aspect of writing the
students pay more attention to seems to indicate that the focus or priority when revising
depends on a number of factors, among them the type of writing assignment and the
course for which the assignment is required. It could be argued that when students work
on an assignment for a literature class, or any other content class for that matter, they
tend to focus their attention on the content and on being able to organize their ideas in an
effective way, whereas when the students work on a writing assignment for a language
class, where word choice, sentence stmcture, and punctuation become fundamental
concems, they tend to put all their efforts towards polishing these formal aspects of
language.
Another issue that was not brought up by the participants in this study but is
worth considering at this point is that some students may often make grammar and
mechanics the focus of their revision efforts because they could be adhering to the advice
received from their language or grammar instructors. Traditional instructional practices
107
in many EFL environments-especially in institutions that train EFL teachers-tend to
emphasize accuracy over meaning or communication, and evidently these practices often
carry over into students' behaviors.
As a conclusion to this section, two explanations can be offered for the students'
difficulty in ascertaining which aspect of language these students prioritize when revising
their written work. On the one hand, it could be argued that these students do not seem to
be very aware of their behavior as writers when revising their work. This explanation,
however, does not seem sufficient to account for the students' ability to articulate what
they do in particular situations. On the other hand, one could also contend—as one
student pointed out—that it is difficuh to separate the various aspects of language and to
focus on only one aspect at a time when revising a piece of writing. It is possible that,
when going over their first written drafts for the purposes of revision, students try to take
into consideration and cover all aspects of their writing simultaneously.
Despite the difficulties pointed out so far, two tentative conclusions can be drawn
from the preceding discussion. The students participating in this study think or believe,
according to the information in the first sur\^ey, that they tend to focus their attention on
the organization of ideas and the grammatical structures of apiece of writing rather than
the ideas themselves or the mechanical aspects of writing, when revising their rough
drafts. Moreover, and in spite of this apparent tendency, students also seem to focus on
different issues when revising their work, depending on the type of assignment they are
working on and on the class for which they are writing the assignment.
108
3.2. Students' Views on Receiving Advice and Suggestions from the
Texas Tech Universitv On-line Writing Center ^Second Survev)
The second survey, which was administered after the students had participated in
the second exchange with the On-line Writing Center consultants, aimed at exploring the
participants' opinions and perceptions about the project in general and about the concept
of receiving feedback from instructors who are NESs, in particular. This second survey
also gauged the degree of the students' receptivity to On-line Writing Center consultants'
suggestions.
Before a more detailed discussion of resuhs, a few general remarks should
be made conceming the information obtained from the answers to the questions in the
second survey. First of all, the students revealed an overall positive attitude toward the
On-line Writing Center responses. They did value the Writing Center consultants' work
and subsequently applied a considerably high percentage of the suggested revisions. A
second point to be made relates to the most important reasons the students incorporated
the recommended textual modifications, among them the possibility of a better grade and
the comments' origin and reasonableness. Third, reflections about some negative aspects
surfaced in this data analysis. Some of these disadvantages pertain to the time delay
between the students' sending and receiving their essays back, their initial apprehension
about having NESs critique their writing, and their uncertainty and uneasiness over
'^ Before answering this questionnaire, the nine students had sent their drafts to and
received comments from the Texas Tech University On-line Writing Center consultants at least
twice. Seven of the nine students indicated that they had sent their compositions three times, and
this information is correct. However, at the time that they answered the survey, they had not yet
received the third response from the On-line Writing Center. Two of the students indicated that
they had sent their rough drafts more than three times. This information is also correct. These
two students had sent drafts that were not required for this class and were therefore not
considered in this study. In spite of this difference in the number of essays sent to the On-line
Writing Center, the fact that the students had participated in the project at least twice makes their
answers comparable and therefore valuable and worth analyzing.
109
computer use. Fourth, despite these few problematic issues, the students' opinions in the
surveys and interviews underscore the comprehensibility and usefulness of the On-line
Writing Center feedback, the students' willingness to work as many suggestions as
possible into their final essays, and their belief in the possibility of improving writing that
these exchanges offer EFL students. Finally, the students believe to have focused their
attention on organization and grammar-related issues-rather than on content or
mechanics—when making changes in their drafts. This perception conceming their focus
of attention when incorporating changes mirrors their perceived tendencies, as expressed
in the first survey.
The nine students indicated that they had incorporated changes upon receiving
comments from the On-line Writing Center consultants. This information is accurate and
was confirmed both by the comparison and analysis of the students' first and final drafts
and through the examination of the interview transcriptions. When asked what
approximate percentage of comments the students had incorporated into their final drafts,
only one student indicated that she had made between 26% and 50% of her textual
changes in response to the On-line Writing Center suggestions. Eight of the nine
students, however, indicated that they had incorporated more than 50% percent of the
On-line Writing Center suggestions for revision. Three of these eight students reported
that they had done so between 51% and 75% of the time. The other five students
reported that they had acted upon the On-line Writing Center consultants' suggestions for
modification between 76% and 100%) of the time. These data, which derive from selfreports, are confirmed by the analysis of the actual modifications observed upon
110
comparison of the students' first and final drafts (see Chapter 4). In 56.06% of the
comments classified for pragmatic intent for the first exchange and in 42.25% of the
comments classified for pragmatic intent for the second exchange, the students acted
upon the On-line Writing Center comments by doing something to their texts (see Table
3.1). If we consider comments classified according to substance or the issue addressed
by them, we observe that 62.86% of the comments for the first exchange and 44.00% of
the comments for the second exchange resuhed in changes in the students' essays (see
Table 3.1). On average, therefore, the actual number of revisions incorporated by these
EFL students in their final drafts (i.e., approximately 50%) match the students' selfreported perceptions in the second survey.
One of the three students who expressed that they had incorporated between 51%
and 75% of the suggestions wrote an explanatory note that reads: "I tried. I don't know if
I did it as I was supposed to." This comment highlights a point that the same student
raised later when being interviewed and is worth considering at this point. This student
explained that a more fluid discussion with the Writing Center consultant about her piece
of writing would have helped her revise her essay more effectively. If this student is
right, this observation underscores the necessity of allowing students access to a
computer classroom not only that permits them to type their work on word-processors, as
they actually did for this project, but also—and more importantly—that gives them access
to the Intemet, and through this medium, to an on-line writing center in an Englishspeaking country. This student's logical and vahd observation also raises the question of
whether emulating and transferring US imiversities' common practices to EFL institutions
111
like the one where this project was conducted would be a good pedagogical move. Such
practices include, for example, instructors' holding individual conferences with students
and faculty and administrative staffs setting up writing centers to assist students with
their writing.
Table 3.1. Comparison of numbers and percentages of changes incorporated by the nine
students according to the types of comments that prompted those changes. Only cases
of complete agreement have been included.
Exchanges: Number of comments classified with
Changes introduced by the 9 students
Complete agreement
Count
First: Comments classified according to
Percent
37
56.06%
30
42.25%
44
62.86%
33
44.00%
Pragmatic Intent: 66 cases
Second: Comments classified according to
Pragmatic Intent: 71 cases
First: Comments classified according to
Substance: 70 cases
Second: Comments classified according to
Substance: 75 cases
Note. These percentages were calculated out of the total number of cases of complete
agreement for each set of data. For example, in the first exchange and for the comments
classified according to pragmatic intent (Taxonomy A) there were 66 cases of complete
agreement; 6 out of 66 comments resulted in "no change," and 23 comments were classified as
"N/A"; therefore, the remaining 37 comments produced some changes, whether these changes
were classified as "positive" or "mixed." The same procedure was used to calculate the rest of the
percentages for the various data. For an explanation of the meaning of "complete agreement" in
this work, the reader is referred to Chapter 4.
Another question in the second survey (question A.4; see Appendix M) sought to
unveil the reasons students had incorporated changes upon receiving comments from the
On-line Writing consultants. The possible options the students could choose from were
112
the following: because you thought you would get a better grade if you incorporated
them; because the suggestions came from a native speaker of English; because the
suggestions seemed reasonable; because the suggestions seemed simple and/or easy to
follow; because the suggestions seemed not to require much time to implement; none of
the above. A count of the answers reveals that the most important reasons the students
chose to introduce the changes suggested by the On-line Writing Center consultants were
the likelihood of obtaining a better grade and the fact that the suggestions were
reasonable. The second most important reason, but the one which yielded more
consensus, was related to the origin or provenance of the suggestions, that is, NESs.
Although only two students indicated that receiving comments from NESs was
the first and the third most important reason for incorporating suggestions, respectively,
six of the nine students chose this reason as a second in order of importance.
Furthermore, three of the students chose the prospect of making a better grade as the most
important reason for incorporating suggestions, while four other students indicated that
the reasonable nature of the consultants' comments was their first priority when making
changes in their texts. It is interesting to note, however, that only one student indicated
that the prospect of obtaining a higher grade was the second most important reason for
incorporating modifications, another student indicated that this reason was fifth in order
of importance, and two students indicated that this reason was the third most important
reason. In other words, there was not much consensus on the significance attached to
grades when these students revealed the prevailing motive for deciding to use writing
center consultants' suggestions. As noted above, four students indicated that the
113
reasonable nature of the comments was the first reason in order of importance for
incorporating changes. Moreover, another four students indicated that the reasonableness
of the comment was second (two students) and third (two students) in order of
importance when deciding to incorporate changes.
Finally, there were other answers conceming the reasons for following the Online Writing Center suggestions. Among the six possible options offered as answers for
question A.4 in the second survey, one student chose the option "none of the above" in
answer to the question. In other words, she did not choose any of the five possible
reasons listed as optional answers. It would have been interesting to find out what other
reasons she may have had for making changes to her essays, if there were any. The fact
that the suggestions were simple and easy to follow was chosen as the third most
important option by two students, and as fourth and fifth options in order of importance
by other two students, respectively. Only three students chose, as a reason for
incorporating a change, the option conveying the idea that the suggestion involved little
time to apply or to carry out. Two of these three students chose this reason as the fourth
most important reason, and one of them chose it as fifth in order of importance.
The most relevant but still tentative conclusion that can be drawn from this
information is that these students valued the suggestions for improvement received from
the On-line Writing Center consultants and decided to incorporate them for three main
reasons: the prospect of receiving a better grade, the reasonable nature of the suggestions,
and the fact that suggestions came from NESs. As six of the nine students chose the
provenance of the feedback as the second most important reason for incorporating
114
changes, and as priority number one for making changes in the students' texts was
divided between the three students and four students who chose "making a better grade"
and "the reasonable nature" of comments as main reasons for incorporating changes,
respectively, it is extremely difficult to conclude, from this limited number of responses,
which was the overriding reason these students decided to incorporate the suggested
changes. As mentioned earlier, the many intemal and extemal variables at play when
subjects in a study follow a particular cotirse of action and the problems associated with
self-reports make it extremely difficult—or even impossible-to assert the validity of these
conclusions.
The preceding information, however, especially the consensus on the value
attached to the experience of receiving comments from NESs, is supported both by the
students' answers to question A.5 in the second survey ("How useful do you think it was
to have an English teacher—native speaker—read your written assigrunents and make
suggestions for improvements before you submitted them to be graded?") and also by the
information obtained from the students' interviews. In answer to the question above, four
students chose the option "extremely useful," and four students chose the option "very
useful." One student left this question unanswered. During the course of the interviews,
even though the students did not specifically answer a question conceming the value or
advantage of receiving feedback from NESs, when referring to the positive aspects of
participating in this project, four of the eight students interviewed highlighted the
importance and relevance of having NESs comment on their writing.
115
As an illustration of this point, one of the students expressed her satisfaction both
to leam that native English speakers had been able to understand her compositions and to
have received positive and encouraging comments from them. Another student noted
that is was good and beneficial to receive comments from these On-line Writing Center
consultants. She explained that she tmsted not only their knowledge of the target
language and but also their insights conceming writing conventions because these
consultants or tutors are both NESs and writing specialists. In a similar fashion, another
student expressed that he considered comments coming from teachers who are NESs to
be more trustworthy than those coming from EFL instructors who are NNESs, especially
in issues related to collocations, prepositions, and idiomatic expressions since NESs are
experts in using them.
Due to the exploratory nature of this study, the survey and interview questions
also inquired about anything that the students considered problematic conceming this
experience of receiving feedback from specialists other than their instructor. The aim of
these questions was to elicit from the surveyed subjects possible solutions to problems
they had experienced during the project and suggestions for improving these electronic
exchanges, in case this project would be eventually carried out with other EFL students in
the same or another institution in the future. Some of the answers to these questions,
therefore, reveal some negative aspects about this e-mail interchange with on-line writing
center consuhants. One of the problematic issues relates to the time elapsed between the
day the students sent their rough drafts and the day they received the feedback and were
therefore able to work on their essays once again. Through informal conversations with
116
the students, I leamed that most of the students were not really affected by the delay, but
a few of them expressed their discomfort at having to go back to a topic, a literary work
that they had covered possibly two weeks previously, when they were now focusing on a
different literary work as the syllabus required. Even though this problem was not voiced
by all the students, the concem about the length of time the students had to wait before
they were allowed to revise their work is a valid one, and it appears pertinent to suggest
that a long delay should be avoided in similar projects in the future.
In the second survey, when asked about their feelings conceming the lapse of time
the students had to wait for a response, most of the students agreed that they felt
"somewhat patient" (five students). The other four students indicated that they had been
"patient" (two of them) and that they had feh "not patient at all" (the other two students).
In other words, the majority of the students' answers (seven) fall into the categories that
describe their feeling as being one of "patience." With regard to anxiety, eight students'
answers fall into the categories "somewhat anxious" (four students) and "anxious" (four
students). Only one of the students indicated that she had feh "extremely anxious."
These figures reveal that most of the students experienced some anxiety but, generally
speaking, were patient when waiting for the On-line Writing Center responses. Even
though these two tendencies may seem somewhat contradictory, anxiety and patience are
not necessarily inconsistent, nor does one emotion preclude the other. Information
obtained from the interviews supports the previous observations. Four of the students
interviewed remarked that the process of working on their compositions, sending them to
the Texas Tech University On-line Writing Center, and then having to wait for a long
117
time before they received feedback was one of the negative aspects of the project, but
they also expressed that the experience as a whole was worthwhile.
The content analysis of the interview transcripts provides insights into some
aspects that students should probably be forewarned about if projects similar to this one
are to be implemented in the future. Three of the eight students interviewed expressed
their negative attitude to this project on related but different grounds. One student
expressed her apprehension or uneasiness at the thought of what the On-line Writing
Center consultants would think of her pieces of writing and language skills, especially if
the consultants knew that the author of these short essays was working toward a degree in
teaching English as a foreign language. Another student expressed that she did not like
the idea of giving her work to a person other than her instmctor because she thought she
usually made serious or embarrassing mistakes. Finally, another student also explained
that her uneasiness about submitting her work to "unknown people," i.e., to a person
other than her instructor, was due to the idea that this other person would probably favor
a different and stricter evaluating criteria. These three students also expressed, however,
that these feelings dissipated after they received the first responses from the On-line
Writing Center consultants. Explaining in more detail what the role of on-line writing
center consultants entails may be advisable as a way to help dispel EFL students'
understandable initial apprehension and uneasiness about an experience like this one.
Even though determining how the students perceived the use of computers was
not one aim of this study, it can be argued from the information gathered both in the
written questionnaires and in the interviews, that most of the students thought that the use
118
of computers was helpfiil in various degrees. Two students indicated that they considered
the use of computers for this project "extremely helpfiil," four students indicated that it
was "helpful," one student indicated that it was "somewhat helpfiil," and only two
students indicated that the use of computers was "not helpful at all." Some participants,
however, also expressed their discomfort or annoyance at having to use computers; four
students, for example, indicated that using computers was "somewhat aimoying," and
two said it was "annoying." Notwithstanding this feeling of annoyance or
disappointment expressed by more than half of the students, three students indicated that
the use of computers was "not annoying at all," and none of the students chose the option
describing computers as "extremely annoying" as an answer.
During the interviews, when being asked about what kind of barriers or obstacles
computers had imposed on their work, all of the students expressed that they did not
consider the use of computers to have been problematic. Three of the students, however,
indicated that they did not particularly hke computers. Two of these students indicated
their apprehension or intimidation at the beginning (when they used a computer for the
first time), and a feeling of uncertainty about the novelty of having to use computers as
part of an in-class assignment. As they also noted, though, with the passing of time, they
began to feel more confident. One of the three students who referred to the
inconvenience of having to use computers explained that computers became useful
eventually, and he appreciated the opportunity he had been given to leam how to use
them. Besides these three students who expressed a kind of negative attitude toward
computers, there was one student who showed a very positive attitude when she
119
explained that, in spite of difficulties such as the lights going out one evening and her
losing part of a document, she had the chance of rewriting her original essay, which she
considered an advantage in itself. To summarize this information about the use of
computers, I would like to note that despite the initial feelings of discomfort and/or
annoyance expressed by seven of the nine students in the second survey, all the students
reflected a generally positive attitude, in both the second survey and the interviews,
toward the help computers can provide, especially after they had used them a couple of
times. The students recognized that is spite of some difficuhies associated with their
rudimentary typing skills and lack of knowledge about technology, it was computers that
had allowed them to interact with NESs.
As far as the students' reception of feedback and general reactions are concemed,
the answers to the questions in section B of the second survey (see Appendix M) reveal
that the majority of the students were satisfied with the experience and the opportunity
they had been given to send their compositions to an English-speaking community for
responses. The nine students expressed that they had understood the On-line Writing
Center responses. Seven students indicated that they had understood between 76% and
100% of the comments, and two students reported that they had understood between 51%
to 75% of the comments. Moreover, the nine students indicated that they did not need the
help of a classmate to understand the comments. Six of them, however, noted that they
had needed the help of their instructor on some occasions. One student included this
remark in her survey answers: "I understood the language, but sometimes I didn't
understand the point." Even though in my role as a researcher I remember some of the
120
students' asking me about the consultants' responses, these questions were mostly related
to whether I-being their instructor and evaluator-agreed with what the consultants had
stated. The students' questions did not seem to be directed to the meaning of comments.
The only issues that some students appeared to be somewhat confused about were
problems conceming comma splices and mn-on sentences, probably because these were
new concepts to most of them.
As revealed by the answers to question B.4 in the second survey, eight out of the
nine students regarded the comments as "useful" (three students) and "extremely useful"
(five students). One of the students who chose the "extremely useful" option added in her
survey this explanatory note: "But I would say that it would have been better if I had had
the chance to have more exchanges or more communication to discuss some topics."
Only one student indicated that the comments had been only "somewhat" useful. His
choice may reflect the fact that this student was among the three more skilled students in
the use of English. In fact, his compositions were always fairly good and needed little
work to be made more effective. Therefore, the comments that this student received may
not have been that helpful to him for the simple reason that he did not receive many
suggestions anyway. This last observation, however, remains tentative, since it is
somewhat risky to draw conclusions from this partial information.
Upon being asked what type of On-line Writing Center cormnents had been
ignored (question B.5, Appendix M), the nine students answered "none." This
unanimous answer indicates that the students not only read but probably tried to
incorporate all types of comments. This tendency is confirmed, as indicated above, by
121
the percentages of suggestions that were in fact acted upon by the students when
rewriting their first drafts both for the first and second exchanges (see Table 3.1). The
attention paid by these EFL students to the comments sent by the On-line Writing Center
consuhants lends support to a more widespread implementation of this innovative
pedagogical practice, which should probably be encouraged and exploited in EFL
institutions that already possess a computer laboratory.
The students were also asked about the frequency with which they had
incorporated different types of suggestions (question B.6, in Appendix M). Priority
number one was assigned to comments conceming both grammar (four students) and
organization of ideas (four students). Only one student indicated that she had
incorporated comments deahng with content with the most frequency. The opinions on
what constituted the second most frequently incorporated comments were somewhat
evenly divided. Two students chose content, three students chose organization of ideas,
two students chose grammar structures, and two students chose mechanics as second in
order of frequency. The third most frequent concem was content (for four of the nine
students), the organization of ideas (for one student), grammar structures (for two
students), and mechanics (for two students). Finally, the fourth most frequently used
comments were those dealing with mechanics (for four students), content (for two
students), organization of ideas (for one student) and grammar (for one student). One
student omitted indicating which language aspect was fourth in frequency of use.
All this information conceming the students' behavior during the project seems to
corroborate the students' usual practices, as self-reported in the first survey. From the
122
information in the first survey, it is apparent that organization of ideas-followed or
matched by grammar concems-constituted one of the main foci of the students'
attention. Third and fourth in order of importance came content and mechanics,
respectively. In response to the second survey questions pertaining to the students'
revision decisions after they had received some feedback from the Texas Tech University
On-line Writing Center consultants, the students consistently revealed priorities similar to
those they had previously indicated in the first survey. Even though global concems such
as the organization and arrangement of ideas seem to occupy an important place in the
students' minds when writing and revising, grammar considerations still play a prominent
role in EFL environments, as these students expressed in the surveys and as observed in
their actual textual modifications or revisions in their final drafts (this issue is discussed
in more detail in Chapter 4).
The answers to the last four questions in the survey (section C in the second
survey; see Appendix M) reflect more of a consensus among these nine EFL students. As
far as the benefits derived from these exchanges are concemed, all the students seemed to
agree that, overall, they had participated in an effective leaming experience. The nine
participants agreed that their writing abilities had improved from receiving and trying to
incorporate feedback. One student, however, wrote this comment in her survey: "I feel
that many things have changed but I would like to know my teachers' [sic] opinion about
this. For me it has been helpful." What this student might have been referring to by this
statement is that the students did not receive a report from their instructor saying whether
123
their compositions had improved from one draft to the other. In other words, she lacked
an extemal opinion to validate what she thought to be true.
The nine students also indicated the possibility of improving their writing by
additional and more frequent electronic submissions. Further, all of them agreed that
they would recommend other students to send their written assignments to NESs in an
on-line writing center, as they had done. Finally, they all expressed that, if the school
gave them the chance to have their written assignments read and commented on by an online writing center consultant who is also a NES, they would go to the trouble of typing
their assignment, sending it, and waiting for a response. Overall, this information
indicates that the experience was viewed as positive by these EFL students. Of course,
the information from surveys and interviews, both of which probe students' perceptions,
is not 100% reliable, but evidence from the actual work the students did while rewriting
their short essays—as will be discussed below—confirms the benefits of allowing and
encouraging students to send their written work to NESs in on-line writing centers for
feedback.
124
CHAPTER 4
RESULTS: TEXAS TECH UNIVERSITY ON-LINE
WRITING CENTER CONSULTANTS' COMMENTS
AND EFL STUDENTS' TEXTUAL REVISIONS
This chapter supplements and expands on the information presented in Chapter 3
by providing a description and analysis of the various types of responses found in the
Texas Tech University On-line Writing Center consultants' feedback and the textual
changes introduced by the group of EFL students in Argentina. This chapter is divided
into three main sections. After a few introductory paragraphs addressing some
methodological issues conceming the population studied for this research, section 4.1
explains the procedure followed to calculate the level of agreement between the two
independent raters who classified both the On-line Writing Center consultants' comments
and the changes in the students' essays. Section 4.2 discusses the frequencies and the
types of On-line Writing Center feedback classified according to pragmatic intent and the
impact, if any, these comments appear to have produced on the students' final drafts.
Finally, section 4.3—following a presentational and organizational pattem similar to that
in the preceding section-examines the frequencies and types of On-line Writmg Center
responses classified according to substance or issue addressed and the apparent effects of
these responses on the students' revisions.
Before a discussion of the level of inter-rater reliabihty in the application of the
three different taxonomies, which will be presented in section 4.1 below, the issue of
125
individual variation in the On-line Writing Center consuhants' work modality and in the
students' language proficiency and language ability should be mentioned. These two
issues deserve some attention because they are essential to the validity of both the data
collected and the data collecting procedures.
On the one hand, it is important to keep in mind not only that the students
participating in this study constitute nine different individuals, but also that there were
five different On-line Writing Center consuhants who responded to the students' essays.
Even though this exploratory study falls under the category of case study, the different
people participating in it are not accounted for individually; that is, neither the
personalities or writing skills—in the case of the students—, nor the work modality or the
various responding abilities—in the case of the On-line Writing Center consultants—have
been traced individually throughout the data. As was earlier indicated, this exploratory
research project sought to observe and report tendencies, not to control or account for a
myriad of variables. It is therefore essential to keep this caveat in mind when interpreting
and evaluating the results reported here.
At least two of the few problems associated with the individual variation
mentioned above should be addressed. In the first place, only one student received
comments from the same consuhant twice, while the rest received responses from two
different writing consultants each time they participated in the exchanges. In addition,
although all of the Texas Tech University On-hne Writing Center consuhants responded
to the students' essays at least twice during the two exchanges, one of the consuhants
responded five times, and two of them responded four times. The remaining two tutors
126
responded two and three times, respectively. These differences, however, will not
receive special attention during the data analysis.
Furthermore, even though there was some unifying element in the nature of the
assignments that allowed for the grouping of the data according to the two exchanges the
subjects participated in, individual variation also played a role in each of the writing
assignments. In other words, although the nine students wrote about one prose work
(Charles Dickens' Great Expectations') in the first assignment, and they later wrote about
poetry (various poems written by three British authors), the students were allowed to
choose from at least two possible topics in each assignment (see Appendix B). In spite of
the fact that individual differences are not analyzed or accounted for here, all these
possibihties for differences and individual variation should be kept in mind by the reader
during the analysis of the results reported in this particular study and also when
comparisons are drawn between the present results and those from previous research.
In order to obtain a more general picture of what happened to the students and
their writing after their participation in this project, the data have been grouped for
analysis. In other words, the group of nine students and the group of five On-line Writing
Center consultants have been studied and analyzed as two units. Consequently,
information about differences between individuals has unfortunately been sacrificed in
favor of more general information. The two groups of participants were, therefore,
deliberately taken as two units with the aim of obtaining greater insight into their
prevailing practices and behaviors. In addition, for the purpose of analysis, the data were
grouped according to the type and date of the assigimient the students had worked on;
127
that is, as data corresponding to the first and to the second exchanges. Furthermore, the
data were also grouped according to the information obtained after the raters applied the
three taxonomies: as data revealing the nature of the comments according to their
pragmatic intent (Taxonomy A); as data establishing the nature of the comments
according to their substance (Taxonomy B); and as data indicating the types of effects or
changes produced in the students' final drafts after some kind of text modification had
been made (Taxonomy C). (See these three taxonomies in Appendix I).
4.1 • Inter-rater Reliabilitv Measures
As a way of introducing the analysis and discussion of the frequency counts and
percentages obtained from the data after they were classified by the two independent
raters, some considerations conceming inter-rater reliability will be noted, and some
general tendencies will be briefly presented at this point. The coefficient of inter-rater
reliability was calculated following Cohen's Kappa (Cohen, 1960; Fleiss, 1973;
Krippendorff, 1980, Landis & Koch, 1977) using the software SPSS Release 6.1 for DEC
Alpha Open VMS. The Cohen's Kappa coefficient was used because this coefficient has
been especially designed to determine the level of agreement on nominal or categorical
data.^^ Cohen Kappa was calculated individually for each exchange; that is, for each
consultant's response and for each student's revised essay. For the reasons indicated
'* Other coefficients normally used with rank-ordered (ordinal) and interval data—but not
appropriate for nominal data—are, for example, the Spearman correlation coefficient (Spearman
rho), the Pearson correlation coefficient, and Kendall Coefficient of Concordance (Kendall's W)
(Hatch, 1991, pp. 448-457, 533-535). These coefficients were therefore not utihzed in this
research since the raters classified nominal data.
128
above, however, it was decided to run the overall Cohen's Kappa coefficient for the
different groups of data. The need to establish some general parameters that would help
report more significant results; the fact that in some cases there was ahnost and even
perfect agreement and in some others the level of agreement was too low; and finally, the
impracticahty of reporting on resuhs for each individual exchange-all of these reasons
combined called for the calculation of the overall Cohen's Kappa agreement coefficient.
For these calculations, the data were grouped according to type of taxonomy used by the
raters and according to the two different exchanges the students participated in. The
overall Cohen's Kappa coefficient was, therefore, calculated for six groups of data, as
follows:
1. Data classified according to the pragmatic intent of the comments for the first
exchange;
2. Data classified according to the pragmatic intent of the comments for the second
exchange;
3. Data classified according to the substance or content of the comments for the first
exchange;
4. Data classified according to the substance or content of the comments for the second
exchange;
5. Data classified according to the effect produced by the change or changes introduced
by the students from the early to the final draft, for the first exchange;
6. Data classified according to the effect produced by the change or changes introduced
by the students from the early to the final draft, for the second exchange.
129
According to the guidelines for evaluating Cohen's Kappa statistics set forth by
Landis and Koch (1977, pp. 164-5) and as the values on Table 4.1 indicate, the level or
strength of agreement between the raters for the different groups of data was "substantial"
in the case of the classification of comments according to the pragmatic intent of
comments (Taxonomy A) for the two exchanges, and in the case of the classification of
students' textual changes (Taxonomy C) for the second exchange. It was "almost
perfect" for the classification of comments according to the issue addressed in them
(Taxonomy B) in both exchanges, and it was "moderate" for the classification of
students' textual changes (Taxonomy C) in the first exchange. All the ratings tasks for
the six groups of data were significant at 0.001. These results, even though acceptable
overall, hide some potential problems observed upon comparison of the two independent
raters' ratings. These problems call for some analysis. Some of these problems, which
will be examined in Chapter 5, "Conclusions," pertain to some limitations conceming the
design of the taxonomies employed and the training of raters. Other difficulties are
addressed immediately below, in the course of the data analysis. The fact that there was
not a perfect level of agreement limits the number of data that can be used to report
results. Notwithstanding this limitation, some conclusions can be drawn, particularly
from the group of data in which complete and partial agreement occurred.
130
Table 4.1. Values for the overall Cohen's Kappa coefficient, probability level, and
confidence interval for the different rating tasks for the six groups of data.
Rating Task
Kappa
Intent of Comments
P
Confidence Interval
0.703
0.0001
0.613 < kpop> 0.793
0.750
0.0001
0.637 < kpop > 0.864
0.817
0.0001
0.727 < kpop > 0.907
0.816
0.0001
0.682 < kpop > .949
0.462
0.0001
0.335 < kpop > 0.589
0.612
0.0001
0.499 < kpop >0725
(First Exchange)
Intent of Comments
(Second Exchange)
Substance of Comments
(First Exchange)
Substance of Comments
(Second Exchange)
Changes from Early to Final Draft
(First Exchange)
Changes from Early to Final Draft
(Second Exchange)
For the sake of data organization and for the facilitation of data analysis, both the
first and second exchanges will be studied first in light of the pragmatic intent of the
comments, and then in light of the types of changes the students introduced in their
essays (Section 4.2). Once the first and second exchanges have been discussed and
assessed in this fashion, they will be subsequently studied, first in light of content or
substance of the comments, and then in light of the changes incorporated by the students
into their final drafts (Section 4.3).
131
4.2. Tvpes of Comments Classified according to Pragmatic
Intent fTaxonomv A) and Students' Textual Revisions
Before I discuss tiie two sets of data in detail, that is, the data corresponding to the
first and second exchanges, respectively, two general observations can be made from the
analyses of the comments classified according to pragmatic intent and of the changes
introduced by the students in their essays. One main observation is that two comment
categories stand out for their high frequency of occurrence when compared to the other
categories in both sets of data: categories 1 and 3. As a reminder, the six categories that
describe comments according to their pragmatic intent are the following: (1) "positive
comment"; (2) "comment indicating the nature of the problem"; (3) "comment indicating
the nature of the problem and suggesting a solution"; (4) "comment indicating the nature
of the problem and providing the exact word or phrase"; (5) "comment directly
suggesting a solution by giving advice"; (6) "comment directly providing an exact word
or phrase." (See Appendix I for a fuller description of categories.) Among these six, the
two categories that occurred most frequently were number (1) "positive comment" and
number (3) "comment indicating the nature of the problem and offering a solution."
Another important general observation that can be made from the analysis of the
two sets of data relates to the type of changes made by the students. The two categories
of textual changes that occurred most frequently were "N/A" and "positive effect," while
the other categories for effects ("No change" and "mixed effect") occurred much less
frequently or did not occur at all, a fact which makes their percentage of occurrence
comparatively lower.'^ The percentages of "positive" and "N/A" effects were roughly
*' Although the categories "negative or negligible effect" and "avoidance of problem"
132
similar in the second exchange, yet "positive effects" clearly outnumbered "N/A" effects
during the first exchange. A similar pattem was observed when textual changes were
studied in relation to the issue addressed in the comments for both exchanges, as will be
shown later in section 4.3 of this chapter. These general observations about tendencies
detected during the analysis of the data are discussed in more detail below.
It should be kept in mind that the changes incorporated into the students' writing
remain the same regardless of the manner in which the comments were initially
classified. In other words, whether the comments were classified according to their
pragmatic intent (Taxonomy A) or their substance or subject matter (Taxonomy B), the
changes produced in the students' final drafts remain the same. Because of this variation
in the classification of types of comments, when textual changes (or effects) are studied
in relation to the groups of comments classified according to one or the other taxonomy,
their proportions will vary accordingly. In other words, the proportions of textual
changes will vary as a result of the variation in the numbers and percentages of types of
comments in the first place. In this section, the numbers and percentages for different
kinds of effects will be analyzed only in relation to the pragmatic intent of the comments.
The various textual changes introduced by the students in relation to the classification of
comments according to substance will be covered later, in section 4.3, when the
classification of comments according to Taxonomy B becomes the focus of attention.
At this point it is necessary to explain what I mean by some terms that will be
employed to describe the data. As can be observed in Table 4.2, the data corresponding
did occur, their occurrence is not considered here because there were no cases of agreement for
either category.
133
to the first exchange show that in 48.89% of cases the raters were in "complete
agreement" on both the classification of comments according to their pragmatic intent
and the classification of types of changes incorporated by the students. In the case of the
second exchange (Table 4.3), the cases of "complete agreement" represent 59.66% of the
original data. When I say "complete agreement," I mean that the independent raters
agreed not only on the classification of the comments according to their pragmatic intent
(Taxonomy A), but also on the classification of types of changes introduced by the
students (Taxonomy C). An example of "complete agreement" is comment #12 of the
first exchange, since both rater A and rater B classified this comment as "prompt alone"
and the corresponding change as "positive effect."
Table 4.2. Numbers and percentages for cases of complete and partial agreement and for
cases of complete disagreement (first exchange)
First Exchange
No.
Percent
Complete Agreement (Taxonomies A and C)
66
48.89
Complete Disagreement
10
7.41
Partial Agreement: (discrepancy in classification of pragmatic intent.
Taxonomy A)
Partial Agreement: (discrepancy in classification of effects.
Taxonomy C)
21
15.55
38
28.15
135
100.00
134
Table 4.3. Numbers and percentages for cases of complete and partial agreement and for
cases of complete disagreement (second exchange)
Second Exchange
Percent
No.
Complete Agreement (Taxonomies A and C)
Complete Disagreement
Partial Agreement: (discrepancy in classification of pragmatic intent.
Taxonomy A)
Partial Agreement: (discrepancy in classification of effects.
Taxonomy C)
71
59.66
6
5.04
15
12.61
27
22.69
119
100.00
Table 4.2, displaying information about the first exchange, also shows that 7.41%
of the data classified by the two raters were in "complete disagreement," while 15.55%
and 28.15% of the data were in "partial agreement." As can be seen in Table 4.3, the data
from the second exchange indicate that in 5.04% of the cases the raters were in "complete
disagreement" after the application of the two taxonomies. The 12.61% and 22.69%)
"partial agreement" correspond, respectively, to discrepancies in the classification of
comments according to pragmatic intent and to discrepancies in the classification of
students' textual changes. By "complete disagreement," I mean that the two raters
classified both a consuhant's comment and the student's corresponding textual change in
a different way. An example of "complete disagreement" is comment #23 of the first
exchange, which was classified as "prompt alone" by rater A, and as "nature of problem
and prompt" by rater B when using Taxonomy A. The corresponding textual change was
also classified differently by the two raters; that is, rater A considered that the change by
135
the student had produced a "negative or negligible effect," while rater B considered the
change to have produced a "positive effect."
Furthermore, when I say "partial agreement," I mean that the raters agreed on one
of the classifications but disagreed on the other. Two different examples of the same
situation are the following: (1) When using Taxonomy A, rater A and rater B agreed on
their classification of comment #14; that is, both of them classified it as "nature of
problem and supply of word." However, they disagreed on the classification of the type
of change according to Taxonomy C because rater A considered the change to have
produced a "positive effect," while rater B considered it to have produced a "negative or
negligible effect"; (2) Rater A and rater B disagreed on the classification of comment
#19 because rater A rated it as "exact word," and rater B labeled it "prompt alone," yet
they agreed on the type of effect the comment had produced, since they both classified
the change as "positive effect," according to Taxonomy C. (See Appendix O for the
classification tables obtained after the raters applied taxonomies A and C to classify the
data corresponding to the First Exchange.)
The data that may reveal more meaningful and reliable information are evidently
the data in which complete agreement between the two independent raters occurred.
Cases of discrepancies, however, should not be overlooked or dismissed altogether
because they could indicate certain tendencies. This analysis, therefore, will focus on the
cases of complete agreement but will also make reference to cases of partial agreement
between the raters.
136
As can be observed from the comparison of Tables 4.4 and 4.5, the two categories
of comments that occurred most frequently in the data from the first exchange are (1)
"positive comment" and (3) "comment indicating the nature of the problem and
suggesting a solution." This tendency is observed whether we consider the cases in
which there was partial agreement (i.e., agreement in the classification of comments
according to their pragmatic intent and discrepancy in the classification of effects), as can
be seen in Table 4.4, or whether we consider just the cases in which there was complete
agreement, as can be seen in Table 4.5. Category 3 ("comment indicating the nature of
the problem and suggesting a solution") occurred on 42 occasions, or 40.38%) of the time,
out of the sum total of cases of complete and partial agreement (Table 4.4), and 20 times,
or 30.30%, out of the total cases of complete agreement (Table 4.5). If one considers
only the cases of complete agreement, it becomes evident that the percentages for
category 3 are roughly equivalent to the corresponding percentages for category 1
("positive comment") and substantially higher than the percentages for the other four
categories (2, 4, 5, and 6) (see Table 4.5).
The next two categories of comments occurring most frequently were categories 6
and 4, occurring respectively 13 (19.70%)) and 7 (10.60%)) times out of the total number
of cases of complete agreement (Table 4.5). Positive comments occurred 23 times, and
they represent 22.12% out of the sum total of cases of complete and partial agreement
(Table 4.4) and 34.85% out of the total number of cases in which there was complete
agreement (Table 4.5). It is important to note, however, that all the comment categories
except for "positive conmient" could be combined because, after all, all of the other
137
comment types-whether belonging to one category or another-aimed at producing a
change in the students' writing. If categories 2, 3, 4, 5, and 6 are combined, some
tendencies become apparent.
Table 4.4. Numbers and percentages for the various types of comments classified
according to Taxonomy A (pragmatic intent of comments), for the first exchange,
out of the sum total of cases of partial and complete agreement combined.
Category number and label
for comment types
Partial and Complete
Agreement
Complete Agreement
Number
Number
Percent
Percent
1 = Positive Comment
23
22.12
23
22.12
2 = Nature of Problem
5
4.81
2
1.92
3 = Nat. Of Probl. + solution
42
40.38
20
19.23
4 = Nat. OfProbl. + word
10
9.62
7
6.73
9
8.65
1
0.96
15
14.42
13
12.50
104
100.00
66
63.46
5 = Prompt alone
6 = Exact Word
Note. The total number of cases including partial agreement (Agreement for Taxonomy
A and Discrepancy for Taxonomy C) and complete agreement (for both taxonomies) is
104.
138
Table 4.5. Numbers and percentages for the various types of
comments classified according to Taxonomy A (pragmatic
intent of comments), for the first exchange, out of the sum
total of cases of complete agreement on both Taxonomies
A and C (i.e., 66).
Category number and label
for comment types
Cases of Complete Agreement: 66
Number
Percent
1 = Positive Comment
23
34.85
2 = Nature of Problem
2
3.03
20
30.30
4 = Nat. Of Problem + word
7
10.60
5 = Prompt alone
1
1.52
6 = Exact Word
13
19.70
66
100.00
3 = Nat. Of Problem + solution
Even though the categories have been defined and differentiated for classification
purposes, categories 2, 3, 4, 5, and 6 seem to share a common end: to help students see
problems in their writing so that they can take care of them in the particular essay they
are working on, or in future writing assignments. Therefore, if these five categories are
combined and compared to the number of positive comments, it becomes apparent that
comments requiring changes outnumbered positive comments (see Table 4.6). Upon
comparing these percentages, we could argue that there were more areas that needed
improvement in the students' essays than there were effective sections, which did not
need modification. This observation, however, does not constitute an unexpected result,
since this project was conducted under the assumption that these EFL students could
benefit from comments addressing weaknesses in their essays.
139
Table 4.6. Nimiber and percentages of positive comments versus other categories of
comments classified according to pragmatic intent, for the first exchange.
Types of comments Total No. of Cases of Complete
(Categories)
and Partial Agreement: 104
Number
Total Cases of Complete
Agreement: 66
Percent
Number
Percent
2 - 6 combined
81
77.88
43
65.15
1 (Positive Comments)
23
22.12
23
34.85
104
100.00
66
100.00
Note. Under "Types of Comments (Categories)," "2-6" refer to the categories of
comments classified according to pragmatic intent (Taxonomy A), as defined in
Appendix I.
Despite the fact that the comments directly requesting changes or merely
suggesting alterations combined outnumber the comments classified as "positive," we
cannot overlook the fact that category 3, "comment indicatmg the nature of the problem
and suggesting a solution," outnumbers each of the other categories individually (see
Tables 4.4 and 4.5). The fact that category 3 prevails over the other types of comments
may indicate a tendency in the responses that the Texas Tech University On-line Writing
Center consuhants sent electronically to these EFL students. It seems that the writing
consultants who participated as respondents in this project tended not only to point out or
explain what the problem in the students' essays was all about, but also to offer a
suggestion for solving it. It was not the purpose of this study to establish the differences
between face-to-face and electronic consultations, and even less to establish similarities
and differences between this particular On-line Writing Center and other writing centers
140
in the USA, yet there are a few observations that can be made after comparing these
figures.
These Texas Tech University On-line Writing Center consultants were sending
feedback via electronic mail. This circumstance did not allow the consuhants to talk to
the students face-to-face and determine from the students' immediate reactions, gestures,
or facial expressions whether the students had understood the point being made or not.
Aware of this limitation, the On-line Writing Center consultants may have decided to
provide more extensive or detailed comments than they would have otherwise, had they
been face-to-face with the students. This reason may help to explain the higher
percentage of comments that provided not only a brief explanation of the nature of the
problem but also a suggestion or course of action to help the student take care of the
problem. Another speculation that can be made relates to the On-line Writing Center
consultants' awareness of the type of students they were responding to. The consultants
had been advised that they were sending their comments to EFL students, whom the
consultants may have suspected or assumed needed more explicit help than NES students
would have needed.
The results obtained from the data corresponding to the second exchange reveal
tendencies similar to the ones mentioned above for the first exchange. If the percentages
of positive comments for both exchanges are compared, however, a slight increase in the
percentage of positive comments is detected in the second exchange (see Table 4.7). This
slightly higher percentage may be indicative of some progress in the writing ability of
these students, progress or improvement which could be associated with the fact that this
141
was the second time that these students had sent their essays to the On-line Writing
Center and also to the possibility that the students may have developed their language
skills in general and their writing skills in particular by the time the second exchange
took place, after half the school year was over.
Table 4.7. Percentages of positive comments for the first and second exchanges out of the
sum total of cases of partial and complete agreement and out of the sum total of cases of
complete agreement (Taxonomies A and C).
Exchanges: Number of
Positive Comments
First: 23
Second: 29
Total Number of Cases of
Partial and Complete
Agreement: No. (percentage)
Total Cases of Complete
Agreement: No. (percentage)
104 (22.12)
66 (34.85)
98 (29.59)
71 (40.85)
Note. "Partial Agreement" refers to cases of agreement for the classification of
comments according to Taxonomy A and cases of discrepancy for the classification of
students' textual changes according to Taxonomy C. "Complete agreement" refers to
cases of agreement for Taxonomies A and C.
Two other explanations for the higher number of positive comments in the second
exchange should be considered. One of them concems the consultants' possible desire to
coimterbalance the high number of comments calling for correction and rearrangement of
ideas with a high number of positive comments. A confirmation of this tentative
explanation goes beyond the scope of this study, since the perceptions and opinions of the
On-line Writing Center consultants were not part of initial research questions. The other
explanation entails the fact that all the data corresponding to the second exchange-as
142
well as those for the first exchange, as was mentioned earlier-were considered and
evaluated as belonging to one group. In other words, it could be possible that most of the
positive comments were directed only to two or three students who had submitted wellwritten essays, in which case the high number of positive comments does not represent a
general tendency. Upon checking every On-line Writing Center response, however, I
can affirm this was not the case since all the students received a fair share of positive
comments. Although we could speculate about many plausible explanations for observed
tendencies, we have to be careful when attributing certain causes to certain results. The
previous observations, therefore, remain mere conjectures that need further investigation
and more detailed analysis.
Even though the percentage of positive comments was comparatively higher in
the second set of data (or second exchange) than in the first set (or first exchange), we
still observe similar tendencies conceming the numbers of the various types of
comments. In the data belonging to the second exchange, the number of comments
classified as "comment indicating the nature of the problem and suggesting a solution," or
category 3, was higher than the number of comments in any of the other categories taken
individually. This observation applies not only when numbers and percentages are
calculated from the sum total of cases of complete and partial agreement combined, but
also when these frequency counts and percentages are calculated out of the number of
cases in which complete agreement occurred (see Tables 4.8 and 4.9). Category 3
occurred 46 times, or 46.94%, out of 98 comments (the total number of cases of complete
and partial agreement combined after the application of Taxonomies A and C). The
143
categories second and third in frequency of occurrence were, respectively, category 5
("prompt alone or comment directly suggesting a solution by giving advice") and
category 6 ("comment supplying the exact word"). These two categories represent,
respectively, 8.16%o and 7.14%) of the comments, out of the sum total of cases of
complete and partial agreement combined (see Table 4.8).
Table 4.8. Numbers and percentages for the various types of comments classified
according to Taxonomy A (pragmatic intent of comments), for the second exchange,
out of the sum total of cases of partial and complete agreement combined (i.e., 98).
Category number and
label for comment types
Partial and Complete
Agreement
Number
Percent
Complete Agreement
Number
Percent
1 = Positive Comment
29
29.59
29
29.59
2 = Nature of Problem
7
7.14
3
3.06
46
46.94
27
27.55
4 = Nat. of Probl. + word
4
4.08
2
2.04
5 = Prompt alone
8
8.16
6
6.12
6 = Exact Word
4
4.08
4
4.08
98
100.00
71
72.45
3 = Nat. of Probl. + solution
Note. The total number of cases including partial agreement (Agreement for Taxonomy
A and Discrepancy for Taxonomy C) and complete agreement (for both taxonomies) is
98.
144
Table 4.9. Numbers and percentages for the various types of
comments classified according to Taxonomy A (pragmatic
intent of comments), for the second exchange, out of the sum
total of cases of complete agreement (i.e., 71).
Category number and label
for comment types
Cases of Complete Agreement: 71
Number
Percent
1 = Positive Comment
29
40.85
2 = Nature of Problem
3
4.23
27
38.03
4 = Nat. of Problem + word
2
2.82
5 = Prompt alone
6
8.45
6 = Exact Word
4
5.63
71
100.00
3 = Nat. of Problem + solution
In addition, if we consider only the number of cases in which there was complete
agreement (i.e., 71 comments), the proportion of comments classified as category 3 is
also substantially higher than the proportion of corrmients classified under other
categories: category 3 occurs 27 times (or 38.03%)), while categories 5 and 6 occur,
respectively, 6 times (or 8.45%)) and 4 times (or 5.63%)) out of the total of cases complete
agreement (see Table 4.9). The difference between the percentages for category 3 and
categories 5 and 6 individually is quite striking and could, with discretion, be attributedas indicated above-to the lack of face-to-face interaction, to the limitations imposed by
electronic communication, and to the special students to whom the comments were
addressed. Furthermore, another factor that could have contributed to this difference may
be related to the taxonomy design. The categories which showed a much lower
145
frequency of occurrence than category 3 may be similar to one another, in which case
combining two or three of them could increase their frequencies and percentages.
So far, the frequencies and types of comments (classified according to pragmatic
intent) that the Texas Tech University On-line Writing Center consultants provided these
group of EFL students in Argentina have been presented and analyzed. Besides studying
the types of comments, however, another issue needs to be considered: the types of
changes and their supposed effects on the students' writing. The textual changes
produced during the first exchange will be examined first, and the textual changes
produced during the second exchange will follow.
Tables 4.10 and 4.11 offer information about the frequencies and percentages of
the various types of effects as classified by the two independent raters for the first
exchange. "Positive effects" outnumbered effects labeled "no change," "mixed effects,"
and "N/A" in those cases where the total number of cases of partial and complete
agreement was taken into consideration and also in those cases where only the data on
which there was complete agreement were considered. There was a total of 55, or
63.22%), positive effects out of the total number of comments classified with both partial
and complete agreement (see Table 4.10). Thirty-six of these changes correspond to
cases of complete agreement, and they represent 54.55%o of the total number of cases in
which there was complete agreement, that is, 66 cases (see Table 4.11).
146
Table 4.10. Numbers and percentages for types of changes ("effects") in students' drafts for
the first exchange in relation to the classification of comments according to pragmatic intent
(Taxonomy A).
Number and Percent for types of
Effects out of the sum total of cases of Complete Agreement
Complete and partial agreement (87)
Number
Partial Agreement
Percent
Positive Effect: 55 or 63.22%
Number
Percent
36
41.38
19
21.83
No Change: 8 or 9.19%
6
6.90
2
2.30
Mixed Effect: 1 or 1.15%
1
1.15
0
0
23
26.44
0
0
66
75.87
21
24.13
Non-Applicable: 23 or 26.44%
Note. Cases of complete and partial agreement have been distinguished in this table.
"Partial agreement" refers to the cases of agreement for Taxonomy C and cases of
discrepancies for Taxonomy A. "Complete agreement" refers to cases of agreement
for both taxonomies.
Table 4.11. Numbers and percentages for types of effects in
students' drafts for the first exchange, out of the sum total of
cases of complete agreement for both Taxonomy A and
Taxonomy B.
Types of effects (or changes)
Out of the Sum Total of Cases
of Complete Agreement: (66)
Percent
Number
36
54.55
No Change
6
9.09
Mixed Effect
1
1.51
23
34.85
66
100.00
Positive Effect
N/A
147
Only 8 (or 9.19%o) of the changes out of the sum total of comments with complete
and partial agreement correspond to the classification "no change." From these 8 cases,
only 6 correspond to cases in which there was total agreement. These 6 comments
represent 9.09%o out of the total cases of complete agreement (see Table 4.11). In
addition, there was only 1 case of "mixed effect." This one case represents 1.15% of the
total number of cases with both complete and partial agreement (see Table 4.10) and
1.51%) out of the cases with complete agreement (see Table 4.11). Finally, 23 of the
effects were labeled "N/A." The number of N/A effects (23) corresponds to 26.44%) of
the 87 cases that include cases of partial and complete agreement and to 34.85% of the
cases in which complete agreement occurred. The information conceming the low
percentage of cases labeled "no change" and the comparatively high percentage of cases
labeled "positive effect" reveals that, upon writing the final drafts, the students made
what appear to be effective textual modifications. Even though there is not enough
evidence to prove a cause-effect relationship between comment received and change
made, it could be argued that, as the changes or alterations introduced by the students
were actually traceable to particular comments, there is a possibility for that relationship
to exist. We can at least speculate, therefore, that after the students received and read the
comments from the Texas Tech Uruversity On-line Writing Center, they took care of a
considerably high proportion of problems in their essays by introducing revisions that
actually emended mistakes and improved certain aspects of their writing.
As far as the second exchange is concemed, the cases in which there was
complete agreement amount to 71. For this set of data, the number of textual changes
148
classified as "N/A" outnumbered by one count the number of "positive effects" (see
Tables 4.12 and 4.13). There were 28 counts for "positive effect" and 29 counts for the
category "N/A."^" On the one hand, these 28 cases of positive effects represent 39.44%
of tiie total number of cases of complete agreement (Table 4.13) and 32.56% of the total
number of comments, regardless of whether there was complete or partial agreement in
the classification of comments (Table 4.12). On the other hand, the 29 N/A effects
represent 40.85% of the total number of cases in which there was complete agreement
(Table 4.13) and 33.72% out of the sum total of cases of partial and complete agreement
(Table 4.12). However, if we add the number of instances of "positive effect" in which
there was complete and partial agreement, the total count is 39 cases of "positive effect"
out of 86 comments, a figure which represents 45.35%) of all the data. Considered in this
fashion (out of the sum total of cases of both complete and partial agreement combined),
the category "positive effect" outnumbers the category N/A (see Table 4.12).
^° In all the cases, the category "N/A" derivedfrompositive comments.
149
Table 4.12. Numbers and percentages for types of changes ("effects") in students'
drafts for the second exchange in relation to the classification of comments
according to pragmatic intent (Taxonomy A).
Number and Percent for types of
effects out of the sum total of cases of Complete Agreement
Partial Agreement
complete and partial agreement (86)
Number
Number
Percent
Percent
Positive Effect: 39 or 45.35%
28
32.56
11
12.79
No Change: 16 or 18.60%
12
13.95
4
4.65
Mixed Effect: 2 or 2.33%
2
2.33
0
0
29
33.72
0
0
71
82.56
15
17.44
Non-applicable: 29 or 33.72%
Note. Cases of complete and partial agreement have been distinguished in this table.
"Partial agreement" refers to the cases of agreement for Taxonomy C and cases of
discrepancies for Taxonomy A. "Complete agreement" refers to cases of agreement for
both taxonomies.
Table 4.13. Number and percentage for types of changes
in students' drafts for the second exchange, out of the sum
total of cases of complete agreement for both Taxonomy A
and Taxonomy C.
Out of the Sum Total of Cases
Types of effects (or changes) of Complete Agreement: (71)
Percent
Number
Positive Effects
28
39.44
No Changes
12
16.90
2
2.81
29
40.85
71
100.00
Mixed Effects
N/A
150
Whether we emphasize the roughly similar proportions of both "positive" and
"N/A" effects when considering just the cases of complete agreement or we emphasize
the difference between "positive" and "N/A" effects when comparing the proportions
calculated out of the sum total of cases of partial and complete agreement, a fact remains:
the total number of occurrences labeled "positive effect" outnumber both the total number
of cases labeled "no change" and the total number of cases labeled "mixed effect." This
difference lends support to the idea that electronic exchanges such as the ones that took
place during the course of this project could benefit students who are developing their
writing skills. This type of interaction with on-line writing center consultants can benefit
EFL students in that the suggestions for improvement may contribute to the students'
awareness and consequent actions directed at remedying problems affecting a particular
piece of writing. Ideally, the awareness raised by the commentary received would be
translated into action. Furthermore, by becoming aware of their individual writing
difficulties and by getting involved in the actual revision process, students would acquire
and develop appropriate revising strategies that they could later reapply when revising
other pieces of writing.
The final part of this analysis focuses on the possible relationship between the
number of positive changes introduced by the students in their essays and the type of
pragmatic intent of the comments that originated the students' textual modifications.
Because the proportion of positive effects was higher than the proportion of effects
labeled as "no change" and "mixed," and also because positive effects are what
instmctors and students alike would logically want to see in revised essays, I will report
151
only on the relationship between the various types of comments and the positive changes
that these comments seemingly produced, or contributed to producing. The type of
comment that appears to have produced more positive changes is category 3, "comment
indicating the nature of the problem and suggesting a solution." Comments classified as
category 3 resulted in 16 (or 44.44%) instances labeled "positive effect" (see Table
4.14). The second category in frequency of occurrence was category 6, "comment
directly providing an exact word or phrase," producing 12 (or 33.33%) textual changes
classified as "positive." Category 4 follows with 6 (or 16.67%)) "positive effects."
Categories 2 and 5 occupy the last place in order of frequency with only 1 (or 2.78%))
"positive effect" each.
Table 4.14. Frequency counts and proportions for the six types of comments
classified according to pragmatic intent resulting in positive effects in the
students' revised drafts for the first exchange.
Types of comments according to
pragmatic intent (category & label)
Total Number of Positive Effects: 36
Number
Percent
'
Cat. 1= N/A
0
0
Cat.2 = Nature of Problem
1
2.78
16
44.44
Cat.4 = Nature of Problem + word
6
16.67
Cat.5 = Prompt alone
1
2.78
12
33.33
36
100.00
Cat.3 = Nature of Problem + solution
Cat.6 = Exact Word
152
The frequencies mentioned above may not mean much because they may only
reflect the proportion of types of comments. As we saw above, category 3 was the
category with higher frequency of occurrence, so it is no surprise that there were more
category 3 comments than other categories tuming into positive effects. Two other points
could be argued, however. On the one hand, as comments labeled category 3 include a
brief explanation or naming of the problem and, at the same time, provide a solution or
suggest a course of action, the students may have been able to implement the
recommendations in these comments more easily than they would have if they had been
prompted through less explicit comments, thus producing some improvement or a
positive effect in their revised essays. On the other hand, we cannot overlook the fact
that there was another category of comments, category 6, that produced 12 positive
changes. Category 6 refers to comments that directly provide the exact word or phrase
needed to make a passage clearer or more effective. Even more so in the case of category
6, it is likely that, upon being given the right word or phrase needed to improve their
essays, students used the suggestion more readily than they would have if the nature of
problem had been simply explained or pointed out (as is the case in category 2) or only a
prompt to solve the problem had been provided by the consuhant (as is the case in
category 5). Coincidentally, these last two categories were linked to only one "positive
effect" each. Based on these observations, I contend that comments that indicate the
nature of the problem and at the same time suggest a course of action for the student to
follow (category 3) and also comments that provide an explicit suggestion as to how to
solve a problem or emend a mistake (category 6) are more likely to produce positive
153
effects in EFL students' writing than other types of comments. On the contrary,
comments that simply explain the nature of the problem or that simply suggest a solution
alone may not work as well for EFL students.
Similar observations can be made conceming the data obtained from the second
exchange. Most of the positive changes made by the students during the second
exchange can be associated with comments labeled "category 3." In fact, 18 comments
(or 64.29 %)) classified as "category 3" produced changes classified as "positive effects"
(see Table 4.15). The second most frequent category of comments that can be linked to
positive effects is category 6 (as was also observed in the first set of data), which appears
to have resuhed into four instances of "positive effect," or 14.29%).
Table 4.15. Frequency counts and proportions for the six types of
comments classified according to pragmatic intent resulting in
positive effects in the students' drafts for the second exchange.
Types of comments according to
Total Number of Positive Effects: 28
Pragmatic intent (category & label) Number
Percent
1=N/A
0
0
2 = Nature of Problem
2
7.14
18
64.29
4 = Nature of Problem + word
2
7.14
5 = Prompt alone
2
7.14
6 = Exact Word
4
14.29
28
100.00
3 = Nature of Problem + solution
As these percentages indicate, the difference between the number of positive
effects produced by categories 3 and 6 is more striking in the data corresponding to the
154
second exchange than in the data corresponding to the first exchange (compare Tables
4.14 and 4.15). All in all however, the higher percentage of positive effects that can be
traced back to comment category 6, when compared to the percentage of positive effects
resuhing from the other five categories of comments, lends support to my contention that
comments not only indicating the nature of the problem but also suggesting a way to
solve it may be more easily grasped or processed and, hence, more often acted upon by
EFL students.
This contention remains a speculation since no correlation calculations were
carried out in order to establish more vahd relationships between types of comments and
types of effects. The chi square correlation coefficient may provide us with more solid
information and thus allow us to make stronger claims. However, due to the exploratory
nature of this study, these calculations have not been made for these data, yet they are
highly recommended for follow-up research. After an analysis of comments classified
according to pragmatic intent and their relationship to the textual changes made by the
students, attention should be directed to the other classification of comments; that is, the
classification of comments according to the issue they address and their effect on the
students' textual revisions. Before the classification of comments according to substance
is examined in the following section, however, brief mention should be made about the
clarity—or lack thereof-of the comments supplied by the Texas Tech University On-line
Writing Center consultants.
The electronic feedback collected and analyzed for this study reveals that these
On-line Writing Center consultants' comments and suggestions were, overall, very clear
155
and to the point. Furthermore, approximately 50% of the comments included statements
and questions intended to encourage the reformulation of ideas and also explanations or
examples of how the student should or could proceed in order to take care of a problem.
The findings of this study, therefore, somehow contradict general tendencies evinced
among ESL/EFL writing instructors when they provide feedback on students' writing.
These reported tendencies (Zamel, 1985) reflect an undue focus on prose decorum rather
than on the content of a piece of writing and the use of comments that tend to be
misleading or difficuh for the students to interpret. In her study "Responding to student
writing," Zamel (1985) demonstrates through a detailed analysis of her data that ESL
teachers' feedback usually consists of vague or confusing comments, cryptic or unclear
directives, and abstract suggestions and warnings that require a considerable amount of
effort on the part of the students who are expected to decipher them. Zamel further
reflects that ESL teachers are often "so distracted by language-related local problems that
they often correct these without realizing that a much larger meaning-related problem has
totally escaped their notice" (1985, p. 86). This was not the case in this exploratory
study, since the students received a balanced combination of clearly stated comments
dealing with both surface-level problems and content-related problems.
There were only very few cases that in some way break this regular pattem of
clearly stated, understandable commentary and that might be considered problematic
comments. However, these instances do not seem to have rendered the comments
obscure or confusing to the students. One such instance involves what appears to be the
mislabeling of the problem in question. For example, instead of identifying the problem
156
as "incorrect, improper, or unnecessary use of the pronoun 'we,'" the On-line Writing
Center consultant provided a long explanation and named the problem "unnecessary shift
in your writing" (essay #7, comment #3, first exchange)."' Yet, in spite of this kind of
misnomer and convoluted explanation, the consuhant also gave a clarifying example at
the end of the comment that compensated for the aforementioned problem. What is
more, the student seems to have understood the comment because she acted upon it
successfully.
In another instance during the second exchange, another On-line Writing Center
consultant seems to have misread or misinterpreted a student's sentence and provided an
unnecessary explanation (essay #2, comment #4, second exchange).'" Even then, in the
last statement of the comment, this consultant recommended that the student doublecheck or consult with the instructor to ensure that clarity of expression would not be
jeopardized. Finally, a few typographical mistakes in the responses to three essays can be
^' In this instance, the student was using the pronoun "we" instead of using a third person
and hence a more formal style. The student's text is followed by the On-line Writing Center
consultant's response:
Student: "If we take Pip's life, from his childhood to his adulthood, we can clearly see how deep
pride and snobbery have influenced his character and personality."
Consultant: "Your opening in this sentence is a bit awkward in that you seem to directiy address
the reader—you may want to avoid this type of unnecessary shift in your writing—it can be
confusing—you might want to try revising this sentence to something like, 'Miss Havisham,
Estella, and Satis House are only the means by which Pip's pride and selfish nature are triggered
and exposed.'—or something like that."
^^ The transcription of the student's text is followed by the On-line Writing Center
consultant's comment:
Student's text: "'The Hollow Men', [sic] by T.S. Elliot, is a long poem written in 1925 and at the
climax of the author's pessimism."
Consultant's response: "...in this first sentence the expression that someone's hfe climaxes is
awkward. Novels have a climax and so do many other literary works, but not lives. You may
want to use a more common expression such as 'the height of his career'-I would recommend
that you check with your instructor on this one to be certain that the phrase is clear."
157
offered as examples of what could be called "problematic comments." These
typographical errors, nevertheless, do not seem to have hindered understanding of the
message in the comments in question. The examples of "problematic comments"
observed in the data, therefore, seem to be tolerable because they were not ubiquitous and
they did not impede comprehension on the part of the reader.
4.3. Tvpes of Comments Classified according to Content or
Substance (Taxonomv B) and Students' Textual Revisions
Before a more detailed discussion of results is conducted, some general
assertions can be made conceming the comments classified according to the issue
addressed in them (or substance) and about the supposed changes these comments
produced in the students' writing. There was a clear difference between the proportion of
comments addressing content and organization-related concems (global problems) and
the proportion of comments addressing vocabulary and sentence-level concems (formal
problems) for the sets of data corresponding to the two exchanges. In the data from the
first exchange, comments addressing formal problems outnumbered those addressing
global problems, while the data from the second exchange reveal just the opposite.
As far as the changes incorporated in the final essays and their relation to the
comments classified according to substance are concemed, the proportion of cases of
"N/A" effect to that of cases of "positive effect" somehow resembles the corresponding
proportion observed when the classification of the comments according to pragmatic
intent was taken into account, as examined earlier in section 4.2. For the data classified
according to substance, both the percentages of "positive" and "N/A" effects were
158
considerably higher than the percentages for the other types of effects (i.e., "mixed" and
"no effect"), as was also observed for the data classified according to pragmatic intent in
section 4.2. Furthermore, "positive effects" outnumbered "N/A" effects in both sets of
data, the data corresponding to the first and second exchanges. (This was not the case for
the data classified according to pragmatic intent, since the category "N/A"-ahhough by
only one count-outnumbered the category "positive effect.") It should be mentioned,
though, that the number of cases labeled "positive effect" for the data classified according
to substance for the second exchange was roughly similar to the number of cases of
"N/A" effects in that "positive effects" outnumbered "N/A" effects by only three counts.
As can be observed in Table 4.16, the data corresponding to the first exchange
indicate that there were 70 cases (or 51.85%) of complete agreement between the raters
for the classification of comments according to substance and the classification of types
of changes incorporated by the students. For the data corresponding to the second
exchange, there were 75 cases (or 63.03%) of complete agreement between the raters (see
Table 4.17). When I say "complete agreement," I mean that the independent raters
agreed not only on the classification of the comments according to their substance
(Taxonomy B), but also on the classification of the textual modifications, or lack thereof,
made by the students (Taxonomy C). An example of "complete agreement" is comment
#6 of the first exchange, since both rater A and rater B classified this comment as
"vocabulary" and the corresponding textual change as "positive effect." (See Appendix P
for the classification tables obtained after the raters applied taxonomies B and C to
classify the data corresponding to the First Exchange.)
159
Table 4.16. Numbers and percentages for cases of complete and partial agreement and
for cases of complete disagreement for the first exchange.
First Exchange
Number
Complete Agreement (Taxonomies B and C)
Percent
70
51.85
3
2.22
Partial Agreement (discrepancy in classification of substance, Taxon. B)
17
12.60
Partial Agreement (discrepancy in classification of effects, Taxon. C)
45
33.33
135
100.00
Complete Disagreement
Note. The sum total and percentage for the two cases of partial disagreement combined
are 62 and 45.93%o, respectively.
Table 4.17. Numbers and percentages for cases of complete and partial agreement and
for cases of complete disagreement for the second exchange.
Second Exchange
Number
Complete Agreement (Taxonomies B and C)
Percent
75
63.03
4
3.36
Partial Agreement (discrepancy in classification of substance, Taxon. B)
11
9.24
Partial Agreement (discrepancy in classification of effects, Taxon. C)
29
24.37
119
100.00
Complete Disagreement
Note. The sum total and percentage for the two cases of partial disagreement combined
are 40 and 33.61%), respectively.
The data corresponding to the first exchange also indicate that only 2.22%o of the
classification was in "complete disagreement," while 45.93%) of the classification was in
"partial disagreement," as shown in Table 4.16. The latter percentage can be broken up
into two: 12.60%) of the total data correspond to cases of agreement in the classification
160
of textual changes (or effects) and discrepancy in the classification of the substance or
content of the comments, while the remaining 33.33%o is made up of cases of agreement
in the classification of the comments' substance and discrepancy in the classification of
effects. Table 4.17, on the other hand, shows that for the second exchange there were 4
cases (or 3.36%) of "complete disagreement" and 40 cases (or 33.61%)) of "partial
agreement." By "complete disagreement," I mean that the raters attached different labels
to both a consultant's comment and the ensuing textual change made by the student; for
example, comment #94 of the first exchange was classified as "vocabulary" by rater A,
while it was classified as "grammar" by rater B. The corresponding textual change was
also classified differently by the two raters; that is, rater A considered it "avoidance of
problem," while rater B considered it a "positive effect." When I say "partial
disagreement," I mean that the raters agreed on one of the classifications but disagreed on
the other. Two different examples of the same situation are the following: (1) both rater
A and B labeled comment #14 of the first exchange as "vocabulary," but disagreed in the
classification of the type of change introduced by the student because rater A considered
the change to have produced a "positive effect," while rater B considered it to have
produced a "negative or negligible effect"; (2) rater A and rater B disagreed in the
classification of comment #5 of the first exchange because rater A labeled it as
"vocabulary," and rater B classified it as a "grammar" comment, but the raters agreed on
the type of effect the comment produced, since they both considered there had been "no
change" in the final draft (see Appendix P).
161
As explained earlier, more meaningful and reliable information can be obtained
from data on which complete agreement between the two independent raters occurred.
Cases in which complete disagreement was observed (i.e., cases in which raters disagreed
on the ratings for both taxonomies B and C) will not be analyzed here. Cases of partial
agreement, however, should not be overlooked or dismissed ahogether since these may
help support or may also make us question tendencies observed from the analysis of cases
in which complete agreement occurred. This analysis of resuhs, therefore, will focus
primarily on the cases of complete agreement for rehability purposes, but it will also
occasionally refer to cases of partial agreement between the raters. In other words,
some information will be provided conceming the frequency counts and percentages of
comments calculated out of the sum total of cases of complete agreement and those
calculated out of the sum total of cases of complete and partial agreement combined. The
latter will serve only as a frame of reference since it would be more appropriate, as
indicated above, to look only at the data on which there was complete agreement.
Out of the sum total of cases of complete and partial agreement combined (87) for
the first exchange, 43 of the changes (or 49.42%) were classified as having "positive
effect," with agreement in both taxonomies. Only 12 comments (or 13.79%) correspond
to positive effects with discrepancy in the classification of type of problem addressed in
the comment (see Table 4.18). Therefore, the sum of all cases of positive effects,
regardless of whether there was complete or partial agreement, totals 55, representing
63.22%o of the sum total of cases of complete and partial agreement (see Table 4.18). Out
of the total number of cases of complete agreement (70) for the first exchange, 43 (or
162
61.43%) of the comments correspond to positive effects in the students' final drafts (see
Table 4.19). Moreover, only 8 (or 9.19%) of the comments out of 87 (the sum total of
cases of partial and complete agreement combined) were classified as having had "no
effect" in the students' final drafts. Six of these comments (or 6.90%) showed complete
agreement, while 2 (or 2.30%) showed disagreement only in the type of problem
addressed in the comment, out of the sum total of cases of partial and complete
agreement. Only one comment was classified with complete agreement, according to
taxonomies B and C, and as having produced a "mixed" effect. Furthermore, 20 (or
22.99%)) of the 87 comments were classified as "N/A" according to taxonomy C, with
complete agreement in both taxonomies, and 3 comments (or 3.45%)) were classified as
"N/A," with discrepancy in taxonomy B (see Table 4.18).
All the details in the paragraph above are intended to provide some background
information for one main observation: If we compare the percentages of cases of
complete agreement for "positive effect," "no change," "mixed effect" and "N/A effect,"
out of the total number of cases of complete agreement, or 70 (see Table 4.19), and out of
the sum total of cases of partial and complete agreement combined, or 87 (see Table
4.18), we can clearly see that the percentage of changes classified as "positive effect" is
higher than the percentages for any other category of textual changes in both instances.
163
Table 4.18. Numbers and percentages for types of changes ("effects") for the first
exchange in relation to the classification of comments according to their substance
(Taxonomy B).
Number and Percentage for types of
effects, out of the sum total of cases of Complete Agreement
Partial Agreement
complete and partial agreement (i.e.,
87)
Number
Number
Positive Effect: 55 or 63.22%
Percent
Percent
43
49.42
12
13.79
No Change: 8 or 9.19%
6
6.90
2
2.30
Mixed Effect: 1 or 1.15%
1
1.15
0
0
20
22.99
3
3.45
70
80.46
17
19.54
Non-applicable: 23 or 26.44%
Note. Cases of complete and partial agreement have been distinguished in this table.
"Partial agreement" refers to the cases of agreement for Taxonomy C and cases of
discrepancy for Taxonomy B. "Complete agreement" refers to cases of agreement for
both taxonomies.
Table 4.19. Numbers and percentages for types of changes
in the students' drafts for the first exchange, out of the sum
total of cases of complete agreement for both Taxonomy B
and Taxonomy C.
Out of Sum Total of Cases of
Types of effects (or changes)
Complete Agreement: (70)
Percent
Number
43
61.43
No Change
6
8.57
Mixed Effect
1
1.43
20
28.57
70
100.00
Positive Effect
N/A
164
As can be observed in Table 4.19, out of the data that were classified v^ith total
agreement (i.e., 70), 61.43%) of the changes introduced by the students correspond to
positive effects, and only 6 comments (or 8.57%)) produced "no change" in the students'
revised versions. This difference between the number of comments producing "positive
effects" and the number of comments producing "no effect" may be due to a number of
factors. This difference in the number and proportions, however, is at least proof that the
students made use of the On-line Writing Center consultants' comments to a considerable
extent. Acting on the consultants' suggestions, in tum, produced a positive effect in the
students' final versions in almost two thirds of the cases of total agreement, as can be
observed in Table 4.19, and in ahnost half the cases for the data encompassing cases of
complete or partial agreement in the raters' classifications, as can be observed in Table
4.18. Even though the percentage for the category "mixed effect" is very low, it should
also be mentioned. From the total number of cases of complete agreement for the first
exchange, 1 (or 1.43%)) of the changes were classified as "mixed" (Table 4.19).
Interestingly enough, there were no cases of raters' agreement for changes classified as
"negative or negligible."
The classification of comments according to substance can be studied individually
and then linked to the types of textual changes made by the students. This type of
analysis would allow us to account for, or at least speculate about, the possible
relationship between types of comments and types of textual changes. In the case of the
first exchange, out of the total number of comments on which complete agreement
occuired for both the comments' substance and the students' textual changes (i.e., 70
165
comments, or 51.85%) of the total number of comments), 9 comments were labeled as
"N/A," 20 comments were classified as "content" comments, 5 as "organization"
comments, 16 as pertaining to grammar, 6 as addressing vocabulary problems, and 14 as
addressing mechanics (see Table 4.20). Table 4.21 shows-out of the total number of
cases of complete agreement-the numbers of comments addressing particular issues (i.e.,
content, organization, grammar, vocabulary, mechanics, and N/A) and the numbers of
types of effects assigned by the raters to the changes incorporated by the students that
could be traced back to each comment category.
Table 4.20. Numbers and percentages for types
of comments classified according to substance
out of the sum total of cases of complete
agreement for taxonomies B and C, for the first
exchange.
Types of
Comments
Content
Number of
Comments
Percent
20
28.57
5
7.14
16
22.86
Vocabulary
6
8.57
Mechanics
14
20.00
N/A
9
12.86
Total
70
100.00
Organization
Grammar
Note. The total number of cases of complete
agreement for both Taxonomy B and Taxonomy
C is 70 (or 51.85%) out of 135 comments.
166
Table 4.21. Number of comments classified according to substance in combination
with number of changes ("effects") for the first exchange, out of the sum total of
cases of complete agreement.
Types of
Comments
N/A
Positive
Effect
Mixed Effect Totals
No Change
Content
9
7
3
1
20
Organization
2
3
0
0
5
Grammar
0
16
0
0
16
Vocabulary
0
5
1
0
6
Mechanics
0
12
2
0
14
N/A
9
0
0
0
9
Total
20
43
6
1
70
The figures in Table 4.21, however, are very small to allow us to make any kind
of claim, yet if we collapse what may be considered similar categories into two main
groups, for example, the group of comments addressing global concems (content and
organization of ideas) and the group of comments addressing formal concems (grammar,
vocabulary, mechanics), we can speak of proportionately higher percentages that may
shed some light on these findings. In other words, these umbrella categories will allow
us to consider some language aspects together, and in so doing we will be able to identify
certain tendencies. Table 4.22, for example, shows the same information as Table 4.21,
but arranged in a different fashion-only three categories of comments have been
featured: global comments, formal comments, and N/A comments.
167
Table 4.22. Numbers and percentages for collapsed categories (derived from comments
classified according to their substance) for the first exchange, out of the total number of
cases of complete agreement on Taxonomies B and C (i.e., 70).
N/A
Comment Types:
No. and percent
Positive Effect
No.
Percent No.
Mixed Effect
No Change
Percent No.
Percent
Percent No.
Global:
25 or 35.71%
11
44.00
10
40.00
3
12.00
1
4.00
Formal:
36 or 51.43%
0
0
33
91.66
3
8.33
0
0
N/A:
9 or 12.86%
9 100.00
0
0
0
0
0
0
As Table 4.22 shows, global problems were addressed 25 times (or 35.71% of the
cases), and formal issues were addressed 36 times (or 51.43%) of the cases), out of the 70
comments that were classified with complete agreement. Clearly, comments dealing with
formal problems outnumbered those addressing global ones. This difference
substantiates previous research, which shows that formal aspects of language such as
grammar and mechanics are more frequently pointed out by instructors than are contentrelated aspects of writing when they evaluate students' essays or when they suggest
revisions (Zamel, 1985; Robb, Ross, & Shortreed, 1986; Cohen & Cavalcanti, 1990;
Leki, 1990; Hedgcock & Lefkowitz, 1994). Earlier research also shows that feedback on
local or structural problems is expected by students (Chastain, 1980; Lalande, 1982;
Walz, 1982; Zamel 1985). The comments labeled as N/A in most, if not all, of the cases
168
in this study referred to positive comments on the part of the consultants and made up 9
(or 12.86%)) out of the total number of cases of complete agreement.
In Table 4.22, we also observe that out of the 25 comments addressing global
problems, 11 of the changes (44%) were classified as N/A, according to Taxonomy C. It
should be remembered at this point that the label "N/A" in the classification table for
changes (or Taxonomy C) refers, m the majority of the cases, to the fact that there was no
expected change because the consuhants' comments were positive." Out of these 25
changes derived from global comments, 10 (or 40%)) were classified as "positive effect,"
only 3 (or 12%) as "no effect," and only 1 (or 4%)) as "mixed effect."
In the same table, it can be observed that, of the 36 comments addressing formal
problems, none of the changes (0%) was classified as N/A. The fact that there were no
changes labeled "N/A" traceable to formal comments upon the raters' application of
Taxonomy C may be explained by the fact that, most of the time, these "N/A" effects
resulted from positive comments. As positive comments usually praise the ideas,
development of ideas, and organization of students' writing—rather than formal aspects of
language such as sentence structure, it is no surprise that there were no cases of textual
changes labeled "N/A" resulting from formal comments. Thirty-three (or 91.66%) of
^^ There was another reason for labeling an effect "N/A." Cases in which the consultants
recommended some action that had not been specified in the original writing assignment were to
be labeled "N/A." In three or four instances in the data obtained from the first exchange, a
consultant requested some change that was not part of the original assignment; for example, the
consultant asked the student to quote from the pertinent source. Unfortunately, although this
particular issue had been addressed during the training sessions, it was apparentiy not cleariy
defined or settled in the minds of the independent raters because one of the raters classified both
the comments and the corresponding changes in the final version as "N/A," while the other rater
classified the comments and the corresponding effects by using labels other than "N/A."
169
these 36 changes were classified as "positive effect," only 3 (or 8.33%) as "no effect,"
and no formal comment was classified as having produced a "mixed effect" {0%). These
results are very revealing, for they support earlier research which has established that
students tend to revise sentence structure, word choice, and mechanics more readily than
they do matters of content (Cohen & Cavalcanti, 1990; Hedgcock & Lefkowitz, 1994;
Porte, 1996). The reasons for this behavior could be many. Logical reasoning and
personal experience, however, support the idea that stmctural problems seem more
tangible or concrete than aspects of content and organization. Therefore, the more
elusive quahty of the latter makes their revision more difficuh than that of the former.
Table 4.23. Numbers and percentages of global and formal comments out of the sum
total of cases of complete and partial agreement combined, for the first exchange.
Comment type: No. cases of
Partial Agreement
Complete Agreement
of complete and partial
agreement combined
Number
Percent
Number
Percent
Global: 52
25
48.07
27
51.92
Formal or structure-related: 54
36
66.66
18
33.33
Note. Cases of complete and partial agreement have been distinguished in this table.
"Partial agreement" refers to the cases of agreement for Taxonomy B and the cases of
discrepancy for Taxonomy C. "Complete agreement" refers to cases of agreement for
both taxonomies, B and C. N/A comments have not been included.
Table 4.23 above shows the numbers and percentages for the tmibrella categories,
i.e., global and formal comments, which resuhed both in agreement upon the
170
classification of comments according to taxonomy B and in discrepancy in the
classification of textual changes according to Taxonomy C, for the first exchange. On the
one hand, 52 comments were agreed upon by the independent raters as dealing with
global issues. Twenty-five of the textual changes resulting from these 52 comments
received the same classification, while 27 of the textual changes deriving from of the
same group of comments received different classification by the two independent raters
upon their application of Taxonomy C. The number of discrepancies (27) in the
classification of textual changes represents 51.92% of the sum total of global comments;
the number of "agreements" on the classification of textual changes (25) represents
48.07% of the sum total of global comments (see Table 4.23). Even though the
percentages for cases of complete agreement and partial agreement in the classification of
textual changes are roughly similar, this similarity is not particularly encouraging or
positive. In effect, a 51.92% disagreement seems to be proportionately high and may be
indicative of the raters' difficulty in tracing and/or attaching a label to global changes.
Evidently, both the detection and classification of global changes entail reliance on more
subjective appraisal parameters than stmctural or formal changes seem to require.
On the other hand, however, only 18 textual changes out of 54 comments dealing
with formal aspects of language showed discrepancy in the raters' classification. This
figure represents only 33.33%) of disagreement in the classification of types of effects, out
of the sum total of formal comments. Once again, this difference in the percentages of
partial agreement (i.e., agreement for Taxonomy B and discrepancy for Taxonomy C), or
51.92%) for textual changes ensuing from global comments and 33.33%) for textual
171
changes ensuing from formal comments, may be indicative of the difficulty involved in
rating effects resuhing from comments addressing global issues. The action of detecting
changes resulting from comments related to stmctural or formal aspects of language,
therefore, seems to be comparatively less problematic and easier to accomplish than the
task of classifying the types of changes ensuing from content-related comments.
Finally, in addition to the information presented so far, a more general
observation can be made conceming the data classified according to substance for the
first exchange. If we consider the total number of positive changes the students made out
of the total number of cases of partial and complete agreement, it becomes evident that
63.22%) of the textual changes were classified as "positive effects," as shown earlier in
Table 4.18. The percentage of comments that produced "no change," however, represents
only 9.19%) out of the sum total of cases of partial and complete agreement. These
figures seem to indicate that, overall, the participants effectively used the comments the
On-line Writing Center consultants sent them. In other words, if we compare the two
percentages above, we obtain a clear picture of the difference between them and can infer
that the comments made some kind of impact on the students' textual revisions. Even
more so, if we consider only the cases in which there was complete agreement-as a more
reliable source of information-and use them to establish a similar comparison, we
observe that the difference between the categories "positive effect" (61.43%)) and "no
change" (8.57%o) is equally striking (see Table 4.19). The fact that more than 50% of the
students' changes were classified as "positive effects" points to the value of feedback or
172
commentary that on-line writing center consultants can provide EFL students in distant
places.
The data corresponding to the second exchange show that there was a 63.03%)
complete agreement between the raters for the classification of comments both according
to substance (Taxonomy B) and according to the types of textual changes incorporated by
the students (Taxonomy C), as shown earlier in Table 4.17. The same data indicate that
only 3.36%) of the classification was in complete disagreement, while 33.61%o of the
classification was in partial disagreement. The latter percentage can be broken down into
two groups: 9.24% of the total data correspond to cases of agreement in the classification
of effects and discrepancy in the classification of the language aspect addressed in the
comment (i.e., substance), while the remaining 24.37% is made up of cases of agreement
in the classification of the substance of the comments and discrepancy in the
classification of effects. Here again, as occurred with the data from the first exchange,
most of the discussion will focus on the cases of complete agreement, and the sum total
of cases of partial and complete agreement combined will be only tangentially addressed.
Table 4.24 presents the frequency counts and percentages for types of effects, out
of the sum total of cases in which there was both complete and partial agreement between
the raters for the second exchange. As this table shows, 31 of the comments (or 36.05%)
correspond to the category "positive effect" (with agreement in the classification of types
of comments and types of effects), and 8 comments (or 9.30%)) correspond to the
category "positive effect" (with discrepancy in the classification of type of problem
addressed in the comment), out of the sum total of cases where complete and partial
173
agreement occurred (86). Only 16 out of the sum total of complete and partial agreement
cases, or 18.60%), were classified as having produced "no change." From these 16
comments, 14, or 16.28%o, represent cases of complete agreement, while the remaining 2
comments, or 2.32%o, correspond to cases of agreement on the effect of the comment with
discrepancy in the classification of the nature of the comment. Only 2 comments (or
2.32%) of the 86 cases) were classified with complete agreement showing "mixed effect."
The number of comments classified as "N/A," 29, constitutes a considerably high
percentage of the data for the second exchange (i.e., 33.72%). The majority of these
"N/A" comments, 28 (or 32.56%o out of 86), represent cases of complete agreement, and
only 1 comment (or 1.16%)) represents a case of partial agreement.
If we consider, however, only the data that were classified with complete
agreement, or 75 cases (see Table 4.25), we observe that 31, or 41.34%), of the comments
correspond to cases of "positive effect." In addition, only 14 comments, or 18.66%o, out
of the 75 comments classified with complete agreement by the raters produced "no
effect" in the students' revised versions. The remaining 2 comments, or 2.66%), were
classified as having produced "mixed effect," and 28 comments, or 37.34%), were
classified as "N/A" with agreement in both taxonomies. Interestingly enough-as
reported also for the first exchange-there were no cases of agreement for changes
classified as "negative or negligible" in the data corresponding to the second exchange.
174
Table 4.24. Numbers and percentages for types of effects or changes (Taxonomy C)
for the second exchange in relation to the classification of comments according to
substance (Taxonomy B).
Number and percent for types of
effects, out of the sum total of
complete and partial agreement
(i.e., 86)
Complete Agreement
Number
Percent
Partial Agreement
Number
Percent
Positive Effect: 39 or 45.35%
31
36.05
8
9.30
No Change: 16 or 18.60%
14
16.28
2
2.32
Mixed Effect: 2 or 2.32%
2
2.32
0
0
28
32.56
1
1.16
75
87.21
11
12.78
Non-applicable: 29 or 33.72%
Note. Cases of complete and partial agreement have been distinguished in this table.
"Partial agreement" refers to the cases of agreement for Taxonomy C and cases of
discrepancy for Taxonomy B. "Complete agreement" refers to cases of agreement for
both taxonomies (i.e., B and C).
If we compare the percentage for the category "positive effect" (41.34%)) with the
percentage for the category "no change" (18.66%), out of the 75 cases in which complete
agreement occurred, we obtain a clear picture of the difference between the two types of
changes. As noted earlier for the first exchange, we can infer that the Writing Center
comments made some kind of positive impact on revision in the second exchange as well.
Even if we rely on the data on which only partial agreement occurred, this difference also
becomes apparent. If we consider the number of "positive effects" out of sum total of
175
cases of complete and partial agreement, we observe that 39, or 45.35%), of changes out
of 86 comments were classified as "positive effects," while only 16, or 18.60%o, of the
comments were classified as having produced "no change" (see Table 4.24).
Table 4.25. Number and percentage for types of changes in
the students' drafts for the first exchange, out of the stmi
total of cases of complete agreement for both Taxonomy B
and Taxonomy C.
Types of effects (or changes)
Out of Sum Total of Cases of
Complete Agreement: (75)
Number
Percent
Positive Effect
31
41.34
No Change
14
18.66
2
2.66
28
37.34
75
100.00
Mixed Effect
N/A
upon comparing the percentage of "positive effect" cases against that of "no
change" cases for the data classified with complete agreement (Table 4.25) and for the
data classified with complete and partial agreement combined (Table 4.24), we observe
that the proportion of changes labeled "positive effect" is higher than that of changes
labeled "no change" in both instances. In fact, the percentages that correspond to the 31
cases of "positive effect," i.e., 41.34% of the data with complete agreement (Table 4.25)
and 36.05%o of the data with both complete and partial agreement (Table 4.24), are
considerably higher than 18.66% and 16.28%), which constitute, respectively, the
proportions of changes labeled "no change," out of the sum total of cases of complete
176
agreement (Table 4.25) and out of the sum total of cases of complete and partial
agreement combined (Table 4.24). This difference between the proportions of textual
changes classified as "positive effect" and "no change," therefore, validates, or at least
speaks favorably of, the revision changes prompted by the Texas Tech University On-line
Writing Center. This statement could be regarded as hasty or premature since the
percentages corresponding to changes classified as "N/A," "negative or negligible effect,"
"avoidance of problem," and "mixed effect" represent about 40%) of the remaining data.
Logically, one may not want to adhere to the above statement blindly, without first
considering what other types of effects the comments produced; therefore, these other
types of effects should also be analyzed before conclusions can be drawn.
The rest of the data are divided between "N/A" effects and "mixed" effects.
Twenty-nine, or 33.72%, out of the total number of cases in which partial and complete
agreement occurred were classified as "N/A," while 2 comments, or 2.32%, were labeled
"mixed effect" (Table 4.24). There was no case of complete agreement for "negative or
negligible effect," nor for "avoidance of problem" either. The frequency counts for the
categories "N/A" and "mixed effect" (i.e., 28 for "N/A" and 2 for "mixed effect")
represent 37.34%) and 2.66%, respectively, out of the sum total of cases with complete
agreement (Table 4.25). Most of the cases labeled "N/A" derive from positive comments.
As this remaining part of the data-though seemingly significant in number-do not
provide much information about the students' revision practices, it becomes necessary to
resort once more to the comparison between cases of "positive effect" and "no change."
As stated earlier, only 18.66% out of the comments on which complete agreement
177
occurred produced "no change" in the students' final versions. Therefore, the percentage
of cases of "positive effect" (41.34% out of the sum total of cases of complete agreement)
remains visibly higher than the percentages for the other categories in general and for the
"no change" category in particular (see Table 4.25). This higher percentage of "positive
effects" seems to indicate here-as it did for the data corresponding to the first exchangethat the students used the comments to their advantage in at least 40% of the cases, a
figure which doubles the percentage for the category "no change."
The classification of comments according to the various issues addressed in them
(or their substance) can be studied separately and then linked to the types of textual
modifications made by the students. This analysis would allow us to establish the
possible relationship between types of comment and types of change. Out of the total
number of cases in which complete agreement for taxonomies B and C occurred for the
second exchange, the two independent raters agreed that 14 of the comments belonged to
the category "N/A," 39 of the comments were "content" comments, 3 were
"organization" comments, 11 dealt with grammar, 3 addressed vocabulary problems, and
5 covered mechanics problems (see Table 4.26). These numbers are rather low and,
consequently, do not allow us to visualize and establish clear tendencies. If we place
what may be considered similar categories into two main groups or umbrella categories,
as indicated above for the first set of data (i.e., the group of comments addressing global
problems such as content and organization and the group of comments addressing formal
problems such grammar, vocabulary, and mechanics), we can speak of percentages that
may allow us to draw some conclusions (see Table 4.27).
178
Table 4.26. Number of comments classified according to their substance in combination
with number of changes ("effects") for the second exchange, out of the sum total of cases
of complete agreement.
Types of
Comments
N/A
Positive
Effect
Content
No Change
Mixed Effect Totals
13
13
12
1
39
Organization
0
3
0
0
3
Grammar
1
8
1
1
11
Vocabulary
0
3
0
0
3
Mechanics
0
4
1
0
5
N/A
14
0
0
0
14
Total
28
31
14
2
75
Table 4.27. Numbers and percentages for collapsed categories (derived from comments
classified according to substance) for the second exchange, out of the total number of
cases of complete agreement on Taxonomies B and C (i.e., 75).
Comment Types:
no. and percent
Percent No.
Percent No.
Percent No.
No.
Mixed Effect
No Change
Positive Effect
N/A
Percent
Global Comments:
42 or 56%
13
30.96
16
38.09
12
28.57
1
2.38
Formal Comments:
19 or 25.33%
1
5.26
15
78.95
2
10.53
1
5.26
14
100.00
0
0
0
0
0
0
N/A:
14 or 18.66%
179
Global problems were addressed by the Texas Tech University On-line Writing
Center consultants 42 times (or 56%) and formal problems were pointed out 19 times (or
25.33%)) out of the total number of cases of complete agreement (or 75), as Tables 4.27
and 4.28 indicate. As we saw when analyzing the data from the first exchange,
comments conceming sentence structure, word choice and mechanics outnumbered
comments dealing with content. This set of data, however, reveals the reverse situation:
comments addressing global concems (42) outnumbered those dealing with formal
problems (19). This difference between the t\\'o sets of data could be attributed to various
causes, none of which can be proved or confirmed by the analysis of the present data.
On the one hand, it could be argued that as the students wrote this assignment later in the
school year, after almost half of the academic year was over, the students may have
undergone some leaming or maturation. The grammar rules, vocabulary items and
mechanics-related concems that may have been pointed out throughout the first months
of classes, not just in the various courses the students were taking but also in the On-line
Writing Center consultants' comments during the first interchange, could have been
internalized or incorporated into the students' interlanguage by then. This could be one of
the reasons that the students were making fewer formal mistakes, and as a resuh, the Online Writing Center tutors were not commenting upon them so frequently.
On the other hand, we could also argue that the second assignment required the
handling of more abstract ideas on the part of the students than the first assignment did
(see Appendix B), and upon not being able to convey them very effectively, the students
wrote essays that lacked a clear thesis statement and that were limited to plot summaries
180
of the poems. These global problems may have called for comments that underscored
content and organization of ideas rather than formal or structural elements. In other
words, comments addressing global problems may have been provided more frequently
by the consuhants than formal or structural comments were simply because the content
and organization had not been adequately handled by the students.
Table 4.28. Numbers and percentages of global and formal comments out of the sum
total of cases of complete agreement (for Taxonomies B and C) and of discrepancy
(for Taxonomy C), for the second exchange.
Comment Types: No. of cases
Complete Agreement
of partial and complete
Number
Percent
agreement combined
Partial Agreement
Number
Percent
Global: 65
42
64.62
23
35.38
Formal or stmcture-related: 25
19
76
6
24
Note. Cases of complete and partial agreement have been distinguished in this table.
"Partial agreement" refers to the cases of agreement for Taxonomy B and cases of
discrepancy on Taxonomy C, and "complete agreement" refers to cases of agreement for
both taxonomies. N/A comments have not been included.
Neither of the two explanations mentioned above, however, not even both of them
together, can be considered the exclusive cause for the higher percentage of comments
addressing global problems. Even though the two situations mentioned above can be
considered influencing factors, there are many other variables, which cannot be
accounted for here, but which may have played a vital role in this difference. These
variables include, among others, the individual students' writing ability, the writing
181
consultants' responding preferences, and the students' difficulty in clearly understanding
and writing about poetry as opposed to understanding and writing about prose.
Some final remarks should be made about the percentages obtained after the
comments were classified according to their effect and grouped into the umbrella
categories of "global," "formal," and "N/A" comments. If we look at Table 4.27 again,
we can see that 16 of the 42 textual changes resuhing from global comments (i.e.,
38.09%)) were classified as "positive effect," only 12 (or 28.57%)) as "no change," and
only 1 (or 2.38%)) as "mixed effect." Furthermore, 13 of the 42 comments addressing
global problems resuhed in 30.96%o of textual changes classified as "N/A." We need to
remember that the label "N/A" denotes different ideas in Taxonomy B and Taxonomy C.
The label "N/A" in Taxonomy B means that the comment was not classifiable under any
of the other five categories because it was meant to praise the work of the student, not
necessarily to address any specific issue such as "content" or "vocabulary."
In the classification table for changes (taxonomy C), however, the label "N/A"
refers to the fact that there was no expected change because the consultant's comment
was positive.""^ Not surprisingly, all the comments classified as "N/A" according to
Taxonomy C had been previously classified as "positive" comments by the consultants
according to Taxonomy A, and they represent 14, or 18.66%, of the total number of cases
of complete agreement (75 cases). The fact that positive comments resuhed in "no
" There could have been other reasons for classifying a comment as "N/A" according to
Taxonomy C. One such reason relates to the consultant's requesting some change that had
apparently not been required by the original assignment (see previous footnote).
182
change" is somehow expected since it is natural that students would often focus on fixing
or eliminating problems rather than on modifying passages that have been praised.
Out of the 19 comments addressing formal concems, only 1 of the corresponding
textual changes incorporated by the students (or 5.26%) was classified as N/A. The fact
that the ntmiber of changes labeled "N/A" was negligible for comments addressing
formal problems indicates perhaps~as pointed out earlier in this section-that most of the
"N/A effect" ratings derive from positive comments. These positive comments usually
praise ideas, development of thoughts, and organization of the piece of writing rather
than formal aspects such as sentence structure. The reason that there was only one "N/A
effect" resulting from formal comments may be, therefore, related to the fact that there
was only one comment that praised some formal aspect of the essay. Finally, 15 (or
78.95%) out of the 19 comments addressing formal aspects of writing resuhed in changes
classified as "positive effect," 2 (or 10.53%) of the changes were classified as "no
change," and only 1 (or 5.26%) textual change was classified as having produced a
"mixed effect."
If we compare the different percentages for "positive effect" by tracing changes
made by the students to the types of comments that presumably triggered these changes
in the students' final drafts, it becomes evident that the students made more positive
changes when applying suggestions conceming the formal aspects of their writing
(78.95% of the cases) than when following suggestions related to their ideas and
arrangement of thoughts (38.09%) of the cases) (see Table 4.27). Once again, these
findings resemble the results reported for the set of data corresponding to the first
183
exchange and serve to substantiate earlier research findings indicating that teachers'
responses and corrections seem more likely to emphasize linguistic accuracy by
identifying and locating awkward sentence structure, wrong word choice, misspellings
and improper punctuation for the student rather than to focus on content-related concems
(Lalande, 1982; Leki, 1990; Robb, Ross, & Shortreed, 1986; Fathman & Whalley, 1990).
Furthermore, we cannot overlook the fact that a considerably high proportion of global
comments (30.95%) resulted in textual changes classified as "N/A" to begin with, while
very few of the formal comments, on the other hand, resulted in changes classified as
"N/A." As category "N/A" was often assigned to comments praising the students' writing
according to Taxonomy A, we can speculate that positive comments, which tended to
result in "N/A" changes, derive primarily from global rather than from formal comments.
Table 4.28 shows the numbers and percentages for the umbrella categories
"global" and "formal" comments, out of the sum total of cases in which both partial and
complete agreement occurred for the data corresponding to the second exchange. On the
one hand, 42, or 64.62%, of the global comments were classified by the raters with
complete agreement upon the application of Taxonomies B and C, while only 23, or
35.38%), of the global comments were classified with partial agreement, that is, with
discrepancy in the application of Taxonomy C. On the other hand, 19, or 76%), of the
formal comments were classified with complete agreement on Taxonomies B and C, and
only 6, or 24%), of the formal comments were classified with partial agreement due to
discrepancies in the application of Taxonomy C.
184
If we compare Tables 4.23 and 4.28, we observe that the percentage of
disagreement in the classification of changes prompted by global comments is
considerably lower for the data corresponding to the second exchange (35.38%)) than the
percentage pertaining to the first exchange data (51.92 %) (compare Tables 4.23 and
4.28). In addition, 6 out of 25 formal comments showed discrepancy in the classification
for textual changes for the second exchange. This figure, which represents 24% of the
cases, is also lower than the corresponding percentage in the data corresponding to the
first exchange (i.e., 33.33%). This last difference could be partly a reflection of the
difference in the proportion of comments for the first and the second exchanges. The
number of comments addressing formal concems was proportionately larger for the data
corresponding to the first exchange (18 out of 54) than for the data corresponding to
second exchange (6 out of 25). However, establishing or discussing the cause of this
difference is not so simple.
The proportionately larger number of discrepancies in the classification of the
types of effects produced by students' revisions in the first set of data when compared to
the number of discrepancies in the classification of effects in the second set of data could
be explained in terms of the leaming process the raters possibly underwent while
classifying the data. In other words, this difference between the data from the first
exchange and the data from the second exchange might be associated with the fact that
the raters themselves-if they followed the classification process chronologically.
185
classifying the first exchange first and the second exchange second^^-may have gone
through a leaming process, applying the taxonomies more consistently towards the end.
If we compare the two percentages for discrepancy-which in Table 4.28 is
identified as "partial agreement"-in the classification of effects for the second set of data,
however, we also see a higher percentage of discrepancy in the classification of the
effects resuhing from global comments (35.38%) than in the classification of effects
resuhing from formal or structural comments (24%) (see Table 4.28). This difference
might indicate that classifying effects deriving from comments related to structural or
formal aspects of language constitutes a comparatively less difficuh task than that of
determining effects produced by content-related comments.
In this section, comments were examined in light of their classification according
to substance (Taxonomy B) and in relation to the types of textual changes introduced by
the students. To summarize the main points discussed in this section, I would like to
highlight one major difference between the data corresponding to the first exchange and
the data corresponding to the second exchange. The percentage of formal comments was
considerably higher than the percentage of global comments for the first exchange, yet
global comments outnumbered formal comments in the second exchange. Among the
possible explanations for this difference, the one that seems most plausible relates to the
possibility that as these EFL students had to deal with more complex concepts and ideas
while writing the essays corresponding to the second exchange, the On-line Writing
" The two independent raters actually did read the material corresponding to thefirstand
second exchanges chronologically, as they indicated upon being asked once they had completed
the rating tasks.
186
Center tutors may have detected more problems at the content and organization levels
than at the surface level, and hence provided more comments addressing the former.
Although it is extremely difficuh to compare the first and the second exchanges
for conclusions about the effects that the comments produced in the students' final drafts,
one fact remains: textual changes classified as "positive effect" considerably
outnumbered textual changes classified as "no change" and "mixed effect" (see Tables
4.19 and 4.25). Therefore, the high percentage of positive effects, when compared to the
percentages of the other two categories, supports the idea that having on-line writing
center consultants participate in the revision stage of EFL students' writing processthough not a magical solution to all the problems that affect EFL writing environmentsappears to be highly beneficial.
A final remark about the students' textual revisions pertaining to both the first and
second exchanges seems appropriate at this point. The nine students participating in
these two exchanges seem to have relied heavily upon the On-line Writing Center
consultants' opinions and suggestions. In other words, on very few occasions did the
students correct or change their drafts on their own, without having been somehow
previously prompted to do so. Students did take care of some misspellings or typos they
found while reading their first rough draft for revision purposes on their own, without
being prompted to do so by the On-line Writing Center consuhants; however, it cannot be
determined from the available data whether the students would have made other types of
changes-changes at the level of content or organization of ideas, for example-had they
187
not been prompted to do so. The students seem to have relied on the consuhants'
comments to a great extent and to have acted upon them accordingly.
On the one hand, it seems that when the comments were positive (positive
comments often referred to content and organization of ideas), the students did not seem
to have bothered to look for other possible problems on their own, except for spelling or
typing problems that may have been more evident to them. When the comments
suggested some kind of modification of a student's writing, on the other hand, it becomes
apparent that the student tried to respond to the suggestion in one way or another. This
tendency to rely on what the writing authority says has also been noted by other
researchers studying EFL/ESL revision practices. Leki (1990, pp. 59-60) and Cohen and
Cavalcanti (1990, p. 175), for example, point out that ESL/EFL students tend to require
and appreciate teacher intervention more often than NES students do.
It remains to be studied whether there was overall improvement in the second or
final draft, after the students had incorporated the consultants' suggestions during both
exchanges. This exploratory study looked into the changes derived from each comment
from each consultant in a locahzed or restricted fashion. In other words, the following
three aspects: the problem or mistake in the students' initial draft; the corresponding
comment from the On-line Writing Center consultant; and the modification, or lack of it,
in the student's final version, were traced and analyzed following a chronological order
from the initial version, through the Writing Center responses, to the final draft.
Improvement was not studied or measured holistically. When analyzing changes and
quality of changes introduced by the students, the raters did not take each essay as a
188
whole. In other words, the raters focused on tracing the individual problem and on the
manner in which each problem pointed out by a Writing Center consultant had been
tackled by the student. If the raters had been asked to evaluate the rough drafts and final
versions holistically, I suspect the results would have been slightly different. I would
even suggest that the number of positive changes that appear to be significant in this
study-probably because they were counted individually-would not make much
difference in terms of the overall quahty of the essays.
This last observation puts into question the whole idea of establishing this kind of
connection in the future. I believe, however, that projects like this one should not be
discouraged or dismissed without further consideration. On the contrary, their repeated
implementation would let us elucidate both benefits and drawbacks, benefits that should
be exploited to the fullest and drawbacks that could be controlled or avoided, or at least
minimized. Many pedagogical benefits can be derived from exchanges like those that
took place during this study. Notwithstanding the observation above, the positive
changes introduced by the students in their final drafts at least indicate that the students
were made aware of common problems, and in some cases, they received useful
information as to how to solve similar problems in the future. At the same time, the
students participating in this project were encouraged to keep on writing by the positive
comments they received.
The last two observations are supported by what the students stated during their
interviews and in their answers to the second survey. On the one hand, the majority of
the students explained that this kind of exchange had been beneficial not only because it
189
had given them the chance to communicate with NESs, but also because it had proved to
be a tme leaming experience that they wished they had had earlier in their study program.
On the other hand, most of the students also expressed their satisfaction with the fact that
their pieces of writing had been understood by NESs and in most cases had received
positive comments that contributed to their self-esteem and self-confidence when writing
in the target language.
190
CHAPTER 5
CONCLUSIONS
This exploratory study's main objective was to enable a group of advanced EFL
students in a Spanish-speaking community to receive feedback via electronic mail from
on-line writing center consuhants in an English-speaking country. Concomitant purposes
associated with this main objective, however, guided and gave shape to this study. In
establishing this type of CMC, the teacher-researcher sought to observe, examine, and
analyze the prevailing attitudes and opinions of the students participating in this
innovative leaming experience. Another objective of this study involved the analysis of
the EFL students' revising behaviors as reflected in the textual changes, or lack thereof,
following these electronic suggestions and responses. At the same time and with the
same degree of emphasis placed on the previous issues, this study intended to
characterize the types of feedback and responses the Texas Tech University On-line
Writing Center consultants are likely to offer when responding to EFL learners' writing.
This chapter recapitulates the main threads of the discussion and analysis
presented in the preceding chapters. In so doing, this chapter also advances the main
conclusions derived from the analysis of the data and elaborates on imphcations both for
the EFL classroom and for future research in the field. This chapter has, therefore, three
objectives: (1) to summarize the initial aims and expectations of this exploratory study
and to indicate the extent to which those aims and expectations were fulfilled; (2) to
highlight the tendencies observed in the students' perceptions and behaviors and the
191
characteristics of the On-line Writing Center consuhants' comments that seem more
relevant in view of existing research and prevailing teaching practices in EFL writing
environments; (3) to reflect on some problems or limitations specific to this research and,
based on these reflections, to discuss some caveats and suggest guideposts for future
research.
5.1. Initial Aims and Expectations of the Studv and the Extent
to Which Thev Were Fulfilled
As expressed earlier, this study aimed at exploring the pedagogical usefulness of
having on-line writing center consuhants contribute to the revision processes of advanced
EFL leamers. While doing so, this study indirectly sought to de-emphasize the
significance still often placed upon students' final written products and their linguistic
accuracy, especially in ESL/EFL environments, as reported in the literature (Raimes,
1983; Zamel, 1985; Leki, 1990; Tmscott, 1996).^*^ In other words, this study intended to
put into practice recommendations such as those proposed by Knoblauch and Brannon
(1984) in the field of NES writing, and by Leki (1990) in the field of ESL/EFL writing.
Knoblauch and Brannon (1984) illustrate the differences between what they call
directive and facilitative commentary and elaborate on the possible effects these two
types of feedback can have on students' revisions. They explain that the purposes of both
types of commentary are different by stating: "The purpose...of facilitative commentary
is to induce the reformulation of texts, the pursuit of new connections and the discovery
^^ Tmscott, who is cited later in this chapter, offers a good review of the literature in
error correction in ESL/EFL writing environments.
192
of richer and more comprehensive meanings. By contrast, the main function of directive
commentary is to make a given text look as good as it can" (Knoblauch and Brannon,
1984, p. 130).
Moreover, these two authors argue that facilitative responses should be favored
over directive responses. They emphasize the importance of negotiation of meaning and
reformulation of texts during the revision stage, negotiation and reformulation that may
not come about if teacher responses are full of "editing prescriptions." They further
contend:
The comments of a facilitative reader are designed to preserve the writer's control
of the discourse, while also registering uncertainty about what the writer wishes to
communicate. The questions posed suggest the possibility of negotiation
between writer and reader, leading to richer insights and more meaningful
communication. Negotiation assumes that the writer knows better than the reader
the purposes involved, while the reader knows better than the writer the actual
effects of authorial choices. The dialogue initiated by the comments (which may
also be sustained by oral conversation) enables the writer to reflect on the
connection between what was meant and what the reader has understood, using
any difference between intent and effect as an incentive to test new choices.
(Knoblauch and Brannon, 1984, p. 128)
These views and recommendations, which pertain to the NES writing field, have been
appropriated and apphed by some ESL/EFL writing specialists, as a way of exploring
innovative response techniques and methodologies.
When discussing some of the directions for providing suitable responses to
ESL/EFL students' writing, Leki relies on some earlier observations made by Knoblauch
and Brannon in the field of NES writers. These two NES writing specialists, Leki (1990)
explains, have examined students' failure to use teachers' feedback successfully by
studying the principles that have guided research in the field and by observing
193
instructional practices where this feedback is provided. Summarizing their views, Leki
(1990) states that a large part of research has deah with teachers' written [my emphasis]
commentary, "without considering the ongoing dialogue between student and teacher" (p.
63). Leki also highlights the importance of a continuous interaction between teacher and
student during the revision process by quoting the two conclusions drawn by Knoblauch
and Brannon: "(1) We need to look not at responses written on final drafts but rather at
responses written on intermediate drafts, and how those drafts are reshaped as a result of
the teacher's comments, and (2) we need to look at the ongoing dialogue between
students and teachers" (qtd. in Leki, 1990, p. 63). This lack of attention paid to the
dialogue between the student and the teacher during the revision stage becomes apparent
in classroom practices and constitutes a gap that has been insufficiently investigated and
has therefore become the focus of many composition researchers recently."
In keeping with the aforementioned views about the relevance of an ongoing
dialogue or conversation during the revision stage, but adding a different perspective or
angle to them, this research study offered the possibility of a dialogue not between
students and their teachers, but between students and writing specialists other than their
instructor. These writing specialists acted as consultants, advisors, and guides in the
writing process-more precisely, during the revision stage-of these EFL students.
Although this dialogue was not "ongoing" as Knoblauch and Brannon (1981)
recommend, because the students had the chance to e-mail each of their essays only once.
^^ There have been many attempts to investigate the revision stage in the writing process
and the roles played by the teacher and his/her oral and written feedback on the students' written
work. The first chapter of this work offers a succinct review of the literature.
194
this first experience could be transformed into an ongoing interaction in a future project
that incorporates more interchanges and that utilizes a larger population of subjects.
In addition to permitting a writing consultant to participate during the revision
process-rather than letting the participation or intervention occur at the end of the
process, when the finished product is not hkely to be altered any further-this project was
conducive to separating the two intricately interwoven roles that the writing instructor is
often required to fulfill. Leki (1990) reflects on how complicated and demanding this
task is for the instructor and on how unfair the situation becomes for the student. She
explains, for example, that "many writing teachers experience intense discomfort when
forced to evaluate students with whom they feel have been collaborating" (Leki, 1990, p.
65). Accordingly, Leki proposes various approaches that might help solve or slowly
reverse this problem. These solutions, which apply to grading systems and writing
classes, would entail, for example, changing the traditional grading system to processbased grading systems—such as the portfolio approach—and transforming the traditional
general writing course into a Writing Across the Curriculum (WAC) program (Leki,
1990).
Among her observations and recommendations, Leki contends that one solution to
modify this incompatibility between the two roles that writing teachers have to adopt and
adapt to involves assigning the role of coach "to the students' classroom teacher and the
role of evaluator to another. The evaluator looks only at final drafts of the students'
writing and does no more than evaluate, writing no comments or corrections at all"
(Leki, 1990, p. 65). As a variation of this suggestion, this project assigned the role of
195
coach to the On-line Writing Center consuhants, while the role of the grader/evaluator
remained with the literature course teacher.
This particular spht of roles between the writing instructor (evaluator) and the Online Writing Center consultant (coach, guide, or real reader), which was easily and
successfully achieved in this project, may entail some difficulties, though. One such
difficulty could be associated with the inadequacy of the feedback offered, especially if a
fluid communication does not exist between the instmctor and the consuhants about the
aims of the assignment, about the long-term goals of the course, or about the writing
instmctor's expectations. In other words, the On-line Writing Center consultants may
lack the knowledge of the course's general objectives or of the specific content or subject
matter dealt with by the instructor in class, and this situation may contribute to generating
inadequate or inappropriate feedback that may mislead the student writers. Nevertheless,
as the Writing Center consuhants had been informed of the aims of the written
assignments the students would be sending and since the consuhants were familiar with
the literary works the students had to write about, these two issues did not constitute a
major problem in this study.
I would like to summarize the above discussion by saying that this exploratory
study has modestly demonstrated that the two recommendations referred to earlier, i.e.,
that writing teachers should pay more attention to the process of writing rather than to the
finished written product and that the role of guide or consuhant should be separated from
the role of evaluator or grader, can be implemented successfully by having stiidents
(EFL students in this case) receive feedback on their written work from writing
196
specialists other than their classroom teachers, during the revision stage. Furthermore,
because this activity allowed the students to put their writing skills to the test with NESs
and because these interactions had, overall, a positive reception on the part of these EFL
students, on-line writing center collaboration and intervention during the writing process
of advanced EFL leamers seems highly recommendable.
5.2. Main Tendencies in the Students' Perceptions and
Behaviors and Main Characteristics of the On-line
Writing Center Consultants' Comments
In this section of the chapter, a summary of the main findings will be presented in
the form of answers to the three research questions that guided this study. It bears
reiterating that this study was an exploratory study of a small group of subjects, and as
such it does not purport to extend or extrapolate its results to other populations. The
conclusions elaborated here only apply to the group of nine EFL students and to the five
On-line Writing Center consultants who participated m the electronic exchanges m this
project. Nevertheless, the observations and findings derived from this exploratory
research project raise interesting questions and offer insights that could serve as a
springboard for further research.
One of the research questions was concemed with the types of comments the Online Writing consultants e-mailed these advanced EFL leamers. As reported in Chapter
4, which deals with the resuhs of this study, the data coded by the two independent raters
when using taxonomy B (see Appendix I) reveal that the proportion of comments
addressing grammatical, lexical, and mechanical concems was higher than the proportion
197
of comments dealing with global concems (comments addressing content and
organization problems) for the first exchange (see Tables 4.20 and 4.23 in Chapter 4).
For the second exchange, however, the reverse was true: the proportion of comments
addressing surface-level concems was lower than the proportion of comments dealing
with meaning-related problems (see Tables 4.26 and 4.28 in Chapter 4). Furthermore, as
indicated earlier, this difference between the percentage of comments addressing global
problems and that of comments addressing stmctural concems might be a reflection of
the written assignments being commented upon. The assignment pertaining to the second
exchange seems to have required that the students address and handle more complex
ideas than the assignment for the first exchange. This is possibly why more difficuhies at
the global level were pointed out by the On-line Writing Center consultants during the
second exchange than during the first exchange. However, another explanation is also
plausible: by the time the subjects had to work on the second assignment, they may have
internalized and, therefore, applied grammar and punctuation mles that they had not been
aware of at the beginning of the school year. As a result, the essays for the second
exchange received fewer comments relating to sentence structure and mechanics.
The same research question conceming types of feedback provided by the On-line
Writing Center consultants to these EFL leamers can be answered from a slightly
different perspective. Using another classification scheme, taxonomy A (see Appendix
I), the independent raters coded the comments according to pragmatic intent. The resuhs
of this classification task indicate that for both the first and the second exchanges, the
type of feedback that appeared with more frequency was described as "comments
198
indicating the nature of the problem and suggesting a solution by giving advice or by
providing prompts" (category 3). The types of comments second in frequency of
occurrence were "comments supplying the exact word" (category 6) and "comments
directly suggesting a solution by giving advice or prompts" (category 5), for the first and
second exchanges, respectively. One possible explanation for the proportionally higher
frequency of occurrence of comments indicating the nature of the problem and suggesting
a solution could be associated with the medium through which the interaction took place.
As has been indicated by several writing center specialists (e.g.. Baker, 1994; Coogan,
1994, 1995a, 1995b; Gmbbs, 1994; Harris & Pemberton, 1995; Kimball, 1997),
conducting on-line writing tutorials through e-mail is very different from conducting
onsite writing tutorials in the actual writing center, where the writing consuhant and
student experience the benefits (and drawbacks) of face-to-face interaction. As the
interaction between tutors and students in this study was asynchronous, precluding
immediate feedback or response at both ends, the On-line Wnting Center consultants
may have felt compelled to provide more detailed and comprehensive commentary in
order to avoid misunderstandings. That is perhaps why most of the responses not only
pointed out what the problem was all about, but also recommended how to deal with it.
Analyzing the perceptions of the On-line Writing tutors, however, was outside the scope
of this study, and therefore the above observation constitutes only a tentative explanation.
The second research question sought to find out what types of comments the
students acted upon with more frequency. Overall, the students appear to have acted
upon most of the suggestions from the On-line Writing Center. However, not all their
199
actions-as reflected in their textual revisions-produced positive changes. Upon a close
examination of the types of changes introduced by the students in their final versions, it
becomes apparent that the proportion of "positive changes" was always higher when the
students had incorporated suggestions conceming formal problems (usage, grammar, and
mechanics) than when they had tried to incorporate suggestions affecting global concems
(see Tables 4.22 and 4.27 for the first and second exchanges, respectively, in Chapter 4 ).
In other words, the percentage of positive changes resulting from surface-level comments
was proportionately higher than the percentage of positive changes prompted by global
comments. The latter observation becomes relevant in the context of an on-going
discussion, in both NES and ESL/EFL writing instmction and writing research circles.
This on-going discussion involves the controversial issue of whether teaching grammar
or providing grammar-based feedback makes a difference in the development of writing
skills.
There is an impressive body of knowledge and research on this long-debated
issue. In a recent article, John Tmscott (1996) provides a thorough review of existing
views and research on this controversy. He also strongly argues against the idea of
teaching grammar explicitly or correcting writing for linguistic accuracy. In order to
build his argument, he cites both numerous studies that support his argument and a few
studies that, according to him, do not provide evidence for the alleged benefits of
grammar correction. Among the studies that Tmscott includes as evidence against
grammar correction-and that also served as sources and background for this study—are
Semke's (1984), Robb, Ross, and Shortreed's (1986), Kepner's (1991) and Sheppard's
200
(1992). The findings reported in all these articles, which sustain Truscott's thesis,
however, do not and cannot speak for particular cases. All of these studies investigated
particular populations in specific settings, with emphasis on different variables; therefore,
Truscott's recommendation that grammar correction be abandoned in L2 writing classes
seems premature and over-reaching.
Among the studies that Tmscott mentions in order to show that they do not offer
enough evidence to support grammar-based instmction and feedback are Lalande's (1982)
and Fathman and Whalley's (1990)—also consuhed for this particular study. Tmscott
summarizes the main findings in these studies by underscoring the lack of evidence they
offer to make a case for the need of grammar correction in L2 writing classes. This
exploratory study of a group of advanced EFL leamers and one on-line writing center
was not originally designed to help tilt the scales towards one side or the other of this
controversy, yet its results may contribute to helping EFL specialists reevaluate and
possibly reaffirm the value of correcting the grammatical and mechanical aspects of
writing in some EFL environments and during certain stages of the writing process.
Other authors have also pointed out the futility of correcting surface-level
mistakes in students' essays. Among them are Hedgcock and Lefkowitz (1994), who
observe that some "empirical evidence based on . . . L2 investigations (viz., Kepner,
1991; Mangelsdorf, 1989) suggests that tradhional, sentence-level 'correction' contributes
only minimally, if at all, to meaningful revision of leamers' text" (145). This statement,
along with Truscott's arguments, could reasonably make teachers wary of the traditional
techniques they often use when commenting or evaluating their students' pieces of
201
writing. From the results of this exploratory research, however, it could be argued that
sentence level correction is not to be avoided or completely abandoned with advanced
EFL students, especially if they are in a teacher-training program, since these students are
often interested in leaming about and improving their writing at all levels. A more
suitable course of action would entail postponing formal or surface-level considerations
to a later stage in the writing process, instead of avoiding or abandoning them altogether.
One more point is worth mentioning in relation to the frequently reported
emphasis placed on grammar and surface-level issues when teachers provide feedback.
Even though focussing on these concems will probably not improve the effectiveness and
overall quality of a piece of writing, EFL leamers usually request that their instmctors
correct their local or formal mistakes. Whether their requests are the result of the
teaching methodology they have been exposed to for several years (Leki 1991a) or the
logical consequence of their wanting to attain certain level of proficiency and accuracy in
the use of the target language (as was the case with the EFL teachers-to-be in this study,
for example), many EFL students seem to aim at producing impeccable compositions,
flawless in grammar, word choice, or mechanics.
Leki (1991a) offers a logical solution to address the problem of the
disproportionate attention often paid to form at the expense of meaning. Leki stresses
that one way to reverse the common situation characterized by ESL students' expecting
local corrections from their teachers is to explain plainly to them-but without neglecting
or overlooking their preferences and needs-what most research has pointed out: that
202
grammar correction is not always effective. She concludes her reflections about this
point by stating:
Ignoring their [the students'] requests for error correction works against the
students' motivation. It would seem, then, that we [composition teachers] must
either accept the students' perceived need to have every error corrected and
accommodate that need, or we must address their preferences directly by
discussing research evidence about the effectiveness of error correction. Those of
us who believe that excessive focus on error correction can be debilitating for
students and pointlessly time-consuming for teachers, must at least consider the
need to explain and defend our versions of how to teach language and writing.
(Leki, 1991a, p. 210)
Leki's insightful observations are vahd and could work successfully, especially with
advanced, adult EFL leamers being trained to become teachers of English. Advanced
EFL leamers who intend to teach the language are presumably in a better position than
children or adolescents, who are at a beginner's level of language leaming, to understand
the rationale behind more innovative teaching methodologies and to take responsibility
for their own leaming. This is probably so due to the former's particular interest and
professional investment in the field of ESL/EFL teaching.
As far as the third research question that guided this study is concemed, that is,
"how receptive are EFL leamers to revision practices prompted by electronic exchanges
with on-line writing center consuhants?", it can be said that the advanced EFL students in
this teacher-training college appreciated all types of comments and believed themselves
to have improved their abilities as writers upon receiving all types of comments.
According to the students' self-reports, they improved their skills as writers and they
leamed from this experience of sending their first drafts, receiving feedback from writing
consultants, and rewriting their short essays. The students did not discriminate between
203
global and sentence level concems when explaining the advantages of the receiving
feedback, yet it is evident, from the count of the different comment types and the
modifications made in the students' essays, that linguistic accuracy played a prominent
role in their textual revisions.
Even though the progress of the students' writing skills cannot be proven by the
information gathered from this study, some of the comments by the students indicate that
they were made aware of some linguistic aspects they had probably not been aware of
before the project took place. For example, comments in the students' interviews along
the lines of "now I know what a comma splice is," or "I will be more careful with
punctuation from now on," are indicators that the students had at least stopped to think
about their writing and about possible ways in which they could improve as writers.
Of course, it would be ludicrous to believe that because a student is aware of a
problem, he/she will automatically avoid it in a future assignment. Nevertheless, the
stage of "being aware" could be, at least, one step in the long processes of both leaming a
second or foreign language and leaming how to write in it. In her study about ESL
students' preferences for error correction, Leki (1991a) explains, when summarizing
some researchers' views that support error correction, that
[Eskey (6)] asks how students will ever improve the accuracy of their writing
without someone pointing out their errors and that Cohen (5) maintains... that
while teacher error correction may not produce a long-lasting improvement in
student writing, self-correction and peer correction do focus students' attention on
errors and resuh in greater control of the written language. (Leki, 1991a, p. 205)
These authors cited by Leki support error correction on the grounds that leamers should
be made aware of problems in order eventually to be able to detect and solve similar
204
problems whenever they occur. If student writers are not shown, at least once, by their
peers, by their teachers, or by writing center consultants, why an expression they have
used does not fit in a particular context, or why a sentence structure is ungrammatical or
awkward and hence needs fixing, it will be difficult or impossible for the students to
recognize these problems in other writing situations; it will be difficuh for them to
become readers and critics of their own written work. As has been shown in this study,
on-line writing center consuhants can help EFL leamers become aware and take control
of their own writing, not just at the sentence-structure level, but also—and more
importantly—at the content and organization levels.
The most positive aspect of this project for the EFL students involved resides in
the unique opportunity they had to have their writing commented on, not just for
linguistic accuracy, but also for global issues such as content and organization. As two
early studies demonstrate (Robb, Ross, & Shortreed, 1986; Zamel, 1985), emphasis on
the correction of grammatical accuracy in ESL compositions seems to be ineffective in
that it does not produce substantial changes in the overall quality of students' essays. The
feedback that the group of EFL leamers received in the present study by no means
emphasized form-related problems at the expense of meaning-related concems, nor did it
address or cover all the problems in the students' essays. The type of feedback that the
On-line Writing Center consuhants provided, however, offered an excellent altemative to
a more traditional response method and a viable way of making students aware of how
they can become better writers.
205
The type of feedback provided by the Texas Tech University On-line Writing
Center could probably help fill in the gaps left by the limited feedback often provided by
ESL/EFL instructors, whose attention tends to be caught by problems at the level of
linguistic accuracy and by mechanical concems and whose feedback often appears only
on the finished product (Zamel, 1985). This study-and others like this one-could serve
to pave the way for a new method of providing responses to students' writing. Projects
like this one could be the first steps in making EFL students and teachers understand that
global concems such as the logical development of a thesis are as important as—if not
more important than—microstructural concems, which can always be remedied at a later
stage, once ideas and concepts have been logically and effectively presented.
5.3. Limitations of this Studv and Caveats
Relevant for Future Research
As stated earlier in this chapter, the EFL leamers in this study did not act upon
comments pertaining to content and organization as successfully as they did when they
incorporated suggestions on local-level concems such as grammar, vocabulary, and
mechanics. One of the reasons for this difference may be related to methodological
problems with the classification of changes rather than with the different nature of the
comments or with students' preferences. The fact that there were more positive changes
in response to local-level comments than changes in response to global comments could
be indicative of problems with the raters' having to classify textual revisions resulting
from comments relating to content and organization. As was discussed in Chapter 4,
despite the thorough training of the two independent raters, classifying changes as
206
positive, mixed, or negative remains an exacting task in that it requires objective thinking
and an acute ability to discem between slightly different perceptions. Some of the
independent raters' comments can serve as an illustration of the demands this activity
placed on these individuals. When referring to a few of the items they had to classifyespecially when using Taxonomy C to classify changes-both independent raters
expressed that these items had demanded "a very tough judgment call" on their part.
Another explanation for the limited number of positive changes resulting from
comments addressing global issues could be the difficulty students face whenever they
work on the focus and development of an essay, two activities that undoubtedly demand
more mental work from the writer. Establishing and keeping a clear focus or trying to
reorganize ideas in a piece of writing usually requires more logical thinking than does
fixing surface-level errors, a task which only requires the application of a grammar or
punctuation rule. Leki offers a similar explanation for this tendency to focus on surface
errors at the expense of meaning-related ones and adds a slightly different perspective to
it when she states:
Since errors in grammar, punctuation, and spelling are concrete, they are
relatively easy to attend to, easier for both student and teacher, than responding to
requests for clarification of an idea or further development or support of a point.
Just as teachers who slave over errors in a student's writing may feel some
personal satisfaction in the evidence of their hard work that a marked paper may
give, students who correct these errors may feel also that their corrections move
them that much farther along the path to complete mastery of English. (Leki,
1991a, p. 209)
However, the students' feehngs of success or mastery in their writing does not constitute
the only explanation available.
207
Still another reason for the neglect of or limited success whh the incorporation of
global changes may be associated with "information overioad." This problem has already
been pointed out by Roen (1989)-among other writing specialists-when he emphasizes
the importance of idea generation and development over the formal concems of writing.
Drawing his observations from the information and views advanced by other researchers,
Roen explains that focusing first on global issues "will prevent 'cognitive overload,' a
short circuit in thinking that comes when trying to attend to too many problem-solving
operations (Newell, 1980; Newell & Simon, 1972) or planning operations (Miller,
Galanter, & Pribram, I960)" (Roen, 1989, p. 200). Therefore, when referring to the
characteristics of appropriate feedback that teachers should provide their students, Roen
recommends proper timing for the different types of feedback so that this feedback can
produce a desirable, positive effect. Roen states:
To alleviate or to eliminate cognitive overload in our novice student writers, our
feedback must help them focus separately on individual parts of rhetorical
problems. They should understand that they do not need to generate and organize
and develop and refine ideas while simultaneously editing for perfect spelling,
punctuation, and sentence stmcture. That is, our feedback must help them
understand that there is a rational and humane reason for refraining from editing
until there are enough well-organized and well-developed ideas to warrant
editing. (Roen, 1989, pp. 201-2)
The students in this study could have experienced a "cognitive overload," since they
received all the comments addressing different issues at the same time. Moreover, as
they had only one chance to work the On-line Writing Center suggestions into their
essays, they were forced to attend to all types of feedback simuhaneously.
Roen's insights, therefore, should be taken into consideration whenever further
research on feedback is conducted and if a similar project is implemented between an
208
EFL institution abroad and an on-hne writing center in the US. Exchanges with on-line
writing center consultants could be set up so that these consultants respond to the first
and to the second drafts of the students' essays, on two different occasions. Ideally, the
first time, only feedback on global issues and rhetorical concems would be provided.
Only once the students have worked on the problems pertaining to meaning and
organization would they send their second versions to the writing center again for a
different kind of feedback. This second time, the on-line writing center consultants
would address only surface-level problems such as ungrammatical constmctions, weak
word choice, and mechanical mistakes.
There may be two problems with the latter suggestion. Further research that
involves the participation of conscientious and responsible on-line writing center
consultants and EFL students who have limited access to e-mail may entail certain
difficulties. On the one hand, this type of research would require certain adjustments in
the modus operandi of the on-line writing center. Having on-line writing center
consultants respond only to formal or microstmctural issues in the second exchange could
mean requiring that they go against their philosophy, which-in very general terms-is
based on the idea of facilitating the expression of thoughts in an effective manner rather
than polishing the surfaces or the looks of a piece of writing. Responding just to formal
or structural concems would entail a violation of on-lme writing centers' pedagogical
purposes, a betrayal of their mission statements, which usually support a dialogic or
Socratic interaction between the writer and the reader and habitually discourage
corrective or unidirectional responses from consultants.
209
On the other hand, this particular research may face certain logistics problems
during its development, especially if the research site lacks the necessary technology to
allow the students to send their essays and receive responses on a daily or weekly basis.
As discussed earlier in Chapter 3, this problem affected the present study. If the EFL
leamers sending their written work do not have direct access to e-mail services either in
their homes or at the educational institution they are attending, the process of sending
their essays and receiving feedback could extend over a considerably long period of time.
This situation could be counter-productive for the students who are writing their
assignments and waiting for a response, and for the instmctor who is supposed to cover a
certain number of units or topics before the end of a semester or end of the school year.
Another question that was not examined in this study but would be worth
exploring in a study with similar characteristics in the future is which types of comments-in terms of pragmatic intent—advanced EFL leamers perceive to be more useful when
revising their texts. The information obtained from this type of research could provide
writing instructors with more insights into students' preferences and needs when revising
their essays. Even though the EFL students in this study, both in the surveys and
interviews, were asked about their priorities when revising their drafts (i.e., whether they
acted more frequently on comments addressing content, organization, grammar, or
mechanics-related problems), they were not asked about their preferences for the type of
comments in terms of their pragmatic intent (see the five categories for pragmatic intent
in Appendix I). Furthermore, this study sought to establish a connection between the
students' actual modifications of their texts and the types of comments that had prompted
210
those changes. In other words, this study examined the actions and behavior of the
students rather than the students' preferences regarding the pragmatic intent of the
comments. It could be assumed that if students acted upon a particular comment by
introducing a change in their drafts, this action could reflect not only their understanding
of the comment and their agreement with it, but also their own preferences.
Nevertheless, this statement remains an assumption since this information about the
students' behavior is incomplete until supplemented by data that indicate what type of
teacher response is favored by the students and what type is balked at.
In their article about leamer receptivity and cognitive effects when receiving
teachers' suggestions, Hedgcock and Lefkowitz (1994) discuss the importance of
studying students' affect in relation to feedback. When referring to studies about teacher
intervention in both LI and L2 composing, these two researchers state: "Mixed findings
suggest that we may need to look beyond written output alone to explore the impact of
teacher intervention as it bears on the mediational processes inherent in constructing and
revising L2 text" (p. 142). As these researchers further indicate, psycho-affective
reactions to different kinds of feedback may
bear directly on the successful applications of procedures leading to rhetorically
and linguistically well-formed text (cf. Leki, 1991). Negative attitudes toward
text creation and apprehension due to unfavorable teacher response have been
shown to inhibit, or to completely stifle, the recursive operations which are so
fundamental to meaningful revision among LI writers (Daly, 1985; Hillocks,
1982) as well as L2 students (Gungle and Taylor, 1989). (Hedgcock &
Leflcowitz, 1994, pp. 145-6)
As mentioned earlier and as Hedgcock and Leflcowitz (1994) explain, this issue of
students' perceptions, opinions, and preferences for certain types of comments is worth
211
pursuing because its examination and analysis could provide guidelines for both EFL
writing instructors committed to the process methodology and for on-line writing center
consultants who often send feedback to a very diverse population.
By knowing what types of feedback to avoid when composing comments and by
phrasing the responses according to what seem to be students' preferences, on-line
writing center consuhants as well as EFL writing instructors would be able to provide
appropriate and effective commentary for text improvement and writer development.
This awareness on the part of the respondents or consultants would prevent them from
providing feedback in a form that may make student writers experience a sense of undue
control on the part of the respondent. Moreover, this knowledge of students' preferences
and needs will discourage writing respondents from giving feedback that may cause
student writers to feel that they are being harshly or unfairly evaluated. In follow-up
research, therefore, not only the changes introduced by the students after being prompted
by on-line writing center consultants, but also the students' beliefs and perceptions on the
comment's pragmatic intent (i.e., how it has been phrased and presented to them and
what its performative force is) should be examined and compared. This analysis will
allow researchers to obtain a more complete picture of students' preferences and reactions
to feedback. Video-taping the students' activities while revising their written work or
using think-aloud protocols, while the students reflect and decide what to do whh the
comments received, could be used as two suitable methods of data collection for this type
of study.
212
In direct relation to the issue of language leaming is another aspect that needs
further exploration: the plausibility of the long-term effects that on-line writing-center
consultants' responses could have on the writing skills of EFL students. As mentioned in
Chapter 4 of this work, it remains to be seen whether the changes in usage, style and
grammar incorporated by the EFL leamers meant an overall improvement in the quality
of the students' texts. It would be interesting to find out, in a follow-up study, whether
there is improvement in the global or overall quahty and effectiveness of a text by
including a holistic evaluation of the participants' essays. This global evaluation would
help provide a fuller picture of how on-line writing center feedback contributes, or not, to
the overall improvement of a text. We have to bear in mind, however, that evaluating
writing improvement is a very difficult, subjective, and daunting task.
Another similar question worth investigating is whether the feedback or
corrections received during one exchange can become intemalized so that the students
will later be able to apply the newly acquired knowledge and skills when revising other
pieces of writing. An idea for a future study that tackles this unexplored aspect would
entail, for example, tracing the development of three EFL leamers at three stages of
language proficiency (for example, beginner's, intermediate and advanced levels). These
subjects would e-mail their composhions to an on-line writing center and receive
feedback at least three times during the first half of an academic year. The same three
students would later self-critique and self-revise another set of essays written during the
second half of the academic year, without the extemal help of an instructor or on-line
writing center consultant. This study would permh tracing key aspects of writing from
213
the time when they were pointed out by the on-line writing center consultants, during the
first three exchanges, to the moment when they are taken care of—or neglected-by the
students during the stages of self-critique, self-revision, and self-editing. This study
would allow researchers to establish the growth, or lack of it, of EFL student writers.
If such a research study were conducted, one important consideration should be
borne in mind, however. When explaining the difference between writer development
and text improvement, Knoblauch and Brannon (1984) affirm the following:
The growth of students as writers is not the same as the improvement of texts.
And though writers can progress in a workshop, their performance as estimated
from the completed writing is not always the best indicator of development.
Moreover, the difficulties of accurately assessing the quality of written texts
makes the connection of performance to growth highly problematic, (p. 151)
In light of Knoblauch and Brannon's cautionary remarks, this future study would have to
be very carefully designed and planned so that improvement can be actually measured.
Before referring to some methodological considerations, I would like to mention
briefly two caveats related to two issues discussed in section 5.1 of this chapter: namely,
the importance of a dialogue or conversation between student writer and on-line writing
center consultant and the equal importance of a close communication between the class
instructor and all the writing consuhants participating in projects like the one that took
place here. If a study similar to this one were to be implemented, measures should be
taken to allow the students the opportunity to correspond, via e-mail, with the on-line
writing consultants at least twice for each essay before that essay is graded. Another
essential measure to be taken consists in having each student e-mail his/her essay to the
same consultant so that the interaction between reader/reviewer and writer may resemble
214
an actual dialogue or conversation-though without the face-to-face component-as
closely as possible.
Furthermore, it would also be relevant and worthwhile for the writing teacher and
the on-line writing center consultants to keep in touch via e-mail before and throughout
the course of the study to avoid misinterpretations or confusion about assignments. A
fluid communication would provide the context and background that the on-line writing
center consultants need in order to work within the general parameters and teaching
objectives set for the class in question. In short, it would be advisable to consider the
feasibility of establishing and maintaining this communication link before a researcher
embarks on a study that involves the participation of two groups of people who do not
possess much information about each other, in order to avoid misunderstandings or
confusion regarding the general aims of the course, the instmctor's expectations, and the
premises of writing assignments.
The latter point, which involves the issues of lack of specific knowledge about
course contents or class discussions and lack of information about course objectives and
assignments, did not constitute a major problem in this study. I emphasize "major"
because there were two minor instances of misunderstanding that could have resuhed in
more serious problems. One such instance took place during the first exchange, when the
students were writing an essay about Charles Dickens' Great Expectations, and a few Online Writing Center consuhants asked the students to support the ideas in their short
essays by quoting information from the text. The students perceived these suggestions as
confusing or misleading since the instructor had not required them to incorporate
215
quotations to prove their points in their assignments. Because of the instructor's presence
while the students were working on their revisions, however, this situation, did not
become a problem. Acting in the role of instructor, I told the students that, instead of
using a quote as suggested by the consultant, they could provide an example in their own
words to refer to an event or situation in the literary work in question that would help
them back up their points.
The other instance occurred when one of the EFL students participating in the
study reported being a little upset at the whole idea of receiving comments from people
who did not know the topics the students had dealt with in class. She explained during
her interview that the consultant who responded to her essay could not possibly have
known the information discussed during previous classes, since the consultant had not
been present during these discussions. Although this student's reaction highlights an
important aspect that should not be dismissed as unimportant, a careful reading of the
student's initial essay and of the consultant's responses reveals that the student seems to
have misread or misinterpreted the On-line Writing Center consultant's comments. The
comments and questions the student received indicate that the On-line Writing Center
consuhant was just trying to make the student reflect again upon what she had written in
her essay and what she had read in the poem by means of "facilitative" comments. In
other words, the consultant was not supplying new, contradictory, or confusing
information to the student; she was only prompting the student writer to go back to the
initial stages of writing, to analyze ideas more carefully, and to express her thoughts in a
clearer and more organized fashion.
216
The student receiving the responses, on the other hand, appears to have
experienced serious difficulties when trying to express the complexity of the ideas
conveyed in the poem she had chosen to analyze and write about. Furthermore, the
student had been absent the day the poem had been discussed in class, and she had relied
on a classmate's notes to study for this writing test. In short, even though this student's
discomfort or uneasiness with this aspect of the interaction underscores a fundamental
point, her uneasiness and defensive reaction remain unjustified because she did not have
a clear conceptualization of the poem to begin with, and this was probably the reason she
had not been able to write a well-focused and well-organized essay. It bears repeatmg,
nevertheless, that a fluid and on-going communication between the two ends of the
interaction becomes essential during asynchronous CMC. The need to share information
on both the topics and aims of assignments between the instructor and those writing
consultants who will be reading the students' texts becomes paramount in an interaction
such as the one that took place in this project.
As far as the study's methodological aspects are concemed, the taxonomies
purposely created for the classification of comments (taxonomies A and B) and for the
classification of textual revisions (taxonomy C) deserve some attention since these are
tools that could be used in follow-up research. Although sometimes taxonomies A and B
were difficult to apply because comments did not always fall smoothly into one category
or another (as revealed by the raters' comments during the training session and after their
rating tasks had been completed), the levels of agreement range from "substantial" to
"almost perfect." The fact that the levels of agreement reached between the raters were
217
quite satisfactory validates these two taxonomies, which could, therefore, be used in
studies that intend to confirm or question the present resuhs by investigating similar
issues. In other words, if trained effectively, only one rater, instead of two, would be
needed to classify on-line writing center comments.
In spite of the fact that the level of agreement obtained by the raters when
classifying comments according to pragmatic intent was good, this taxonomy could be
modified to include other aspects such as mode ("directive commentary" versus
"suggestions," and "open" versus "closed questions"), an issue that was studied by Straub
(1997) in his study of NES first year college students' feedback preference. Another
modification of taxonomy A could involve the incorporation of the linguistic features of
comments (questions, statements, imperative statements, or exclamations) in combination
with their pragmatic intent, as has been captured in the taxonomy created by Ferris
(1997) for her study of teacher comments on ESL freshman students at a public
university in the US. Another viable possibihty that would permit the observation of
more factors than the ones observed in this study entails the design of a completely new
taxonomy that would include Straub's (1997) and Ferris' (1997) categories. This
taxonomy could either supplement or completely replace the one used in this study. All
or some of these modifications could be used in order to study not only the types of
comments that on-line writing center consuhants are hkely to provide, but also to inquire
about the students' preferences for each type.
With respect to Taxonomy C, which was employed to classify the changes in the
students' final drafts, the independent raters attained a "substantial" level of agreement for
218
the classification corresponding to the second exchange, but only a "moderate" level of
agreement for the classification corresponding to the first exchange. As stated in the
chapter dealing with the results of this study, the rather low level of agreement between
the raters for the first exchange (K=0.462) and the considerably higher level of agreement
between the raters for the second exchange (K=0.612) could be explained on at least two
grounds. First, the difference could be associated with lack of proper training of the
independent raters. However, this explanation could seem contradictory or inapplicable
if we used it to account for the resuhs from the other two taxonomies (A and B), which
yielded "substantial" and "almost perfect" levels of agreement. In other words, the
explanation based on the possibility of the raters' lack of proper training does not account
for the better level of agreement obtained for taxonomies A and B. Second, the increase
in the level of agreement from the first to the second exchange, upon the application of
taxonomy C, could be related to a leaming process undergone by the raters while using
the taxonomy.
Finally, if we compare the agreement levels for the three taxonomies, we can
clearly see that the values of agreement for students' textual revisions for both the first
and second exchanges (K=0.462 and K=0.612, respectively) were considerably lower
than the agreement levels for the classification of the On-line Writing Center consultants'
comments for taxonomy A (K=0.703 and K=750 for the first and second exchanges,
respectively) and for taxonomy B (K=0.817 and K=816 for the first and second
exchanges, respectively). Although there may be others, a plausible explanation for the
considerably lower agreement in the ratings for taxonomy C, when compared to those of
219
taxonomies A and B, could be the apparent difficulty involved in rating such subjective
issues as "quality" and "improvement" in a piece of writing.
Although the problems surrounding the application of taxonomy C do not render
this taxonomy unusable, the taxonomy's validity as a classifying tool needs to be tested
again. In other words, for a research study that would require the classification of textual
changes in students' subsequent drafts, two courses of action should be contemplated. On
the one hand, two new raters could be trained, with the researcher using more examples
and discussing in detail why the selection of one classification category is better than the
selection of another one. Examples to be discussed could be extracted from this study,
especially those cases that offered the most difficulty in classifying because they resulted
in diametrically opposing categories such as "positive change" and "negative change."
More extended training sessions, along with more discussion of specific examples, could
help raters concur more often on their classification. In addition, a taxonomy that
evaluated the pieces of writing hohstically could be designed and applied in conjunction
with the one used for this study. This additional taxonomy could incorporate a holistic
evaluation measure so that the raters or evaluators would focus on and compare the
general impressions of the first and final drafts, rather than concentrate on detecting and
evaluating more localized changes.
In closing, it would be relevant to highlight what appear to be the most beneficial
and promising aspects of allowing EFL students to establish contacts with NESs in an online writing center. First, the possibility of having a writing speciahst participate in a
facilitative rather than on a controlling or directive manner has been favorably received
220
by these EFL students. There is no reason to believe that other EFL students in a similar
situation would not welcome this opporttmity as well. The good reception on the part of
these EFL students may power the momentum of the motivating force that will encourage
them to keep on writing and to keep on trying to improve as writers.
Second, even though it is not totally clear or definite from the findings in this
study what types of comments—in terms of substance and pragmatic intent—contribute to
the development of EFL students as writers, none of the comments offered by the
instructors seem to have been counter-productive. All of the changes introduced by the
students produced "positive changes," "mixed changes," or "no changes," but none of the
comments resulted in "negative changes." In other words, the number and types of
revisions made by the students seem to indicate that the On-line Writing Center
consultants' comments were not only understood, but also successfully acted upon by the
students. Therefore, the whole experience produced results that can be considered
beneficial for the purpose of encouraging students to revise their written work.
Finally, it is essential that both writing instructors and researchers recognize the
fundamental role that an individual's intent, motivation, and preferences play in any
leaming process. No matter how much instructors control a teaching environment or how
much researchers control and keep track of the many variables that may affect the
development of a student as a writer, it is ultimately the students, as individuals, who
decide—given the necessary conditions—what and when they are ready to leam and
whether they are willing to improve their skills in a second or foreign language.
221
BIBLIOGRAPHY
Adipattaranun, N. (1992). An examination of the variables in the writing process of
ESL/EFL students' in a process-oriented freshman composition course.
Dissertation Abstracts Intemational 53. [Online], 1826.
Aly, M. S. (1992 ). A descriptive study of four EFL teachers' treatment of writing
errors and their feedback in an Arab country. Dissertation Abstracts Intemational,
53, [Online], 0429.
Asiri, I. M. (1997). University EFL teachers' written feedback on compositions and
students' reactions Dissertation Abstracts Intemational. 59, [Online], 0006.
Auerbach, E. R. (1991). Politics, pedagogy, and professionahsm: Challenging
marginalization in ESL. College ESL, l(\), 1-9.
Baker, J. S. (1994). An ethical question about on-line tutoring in the writing lab. The
Writing Lab Newsletter, 18 (5), 6-7.
Barker T. T., & Kemp, F. O. (1990). Network theory: A postmodem pedagogy for the
writing classroom. In C. Handa (Ed.), Computers and communitv: Teaching
composition in the twenty-first centurv Cpp. 1-27). Portsmouth, NH:
Boynton/Cook.
Barker, T. T. (1990). Computers and the instructional context. In D. H. Holdstein & C.
L. Selfe (Eds.), Computers and writing: Theory, research, practice (pp. 7-17).
New York: The Modem Language Association of America.
Barson, J., Frommer, J., & Schwartz, M. (1993). Foreign language leaming using email
in a task-oriented perspective: Interuniversity experiments in communication and
collaboration. Joumal of Science Education and Technology, 2, 565-584.
Batson, T. (1993a). ENFI research. Computers and Composition, 10, 93-101.
Batson, T. (1993b). The origins of ENFI. In B. C. Bruce, J. K. Peyton, & T. Batson
rEds.l Network-based classrooms: Promises and reahties (pp. 87-112). New
York: Cambridge University Press.
Belcher, D., & Braine, G. (Eds.). (1995). Academic writing in a second language:
Essays on research and pedagogy. Norwood, NJ: Ablex Pubhshing Corporation.
Bizzell, P. (1990). Beyond anti-foundationalism to rhetorical authority: Problems
defining "cultural literacy." College Enghsh, 52, 661-675.
222
Blythe, S. (1997). Networked computers + writing centers = ? Thinking about
Networked computers in writing center practice. The Writing Center Joumal, 17
(2), 89-109.
Braine, G. (1997). Beyond word processing: Networked Computers in ESL writing
classes. Computers and Composition. 14. 45-58.
Brandl, K. K. (1995). Strong and weak students' preferences for error feedback options
and responses. The Modem Language Joumal 79. 194-211.
Brannon, L., & Knoblauch, C.H. (1984). A philosophical perspective on writing centers
and the teaching of writing. In G. A. Olson (Ed), Writing center theory and
administration (pp. 36-47). Urbana, IL: NOTE.
Brock, M. N. (1993). A comparative study of computarized text analysis and peer
tutoring as revision aids for ESL writers. Dissertation Abstracts Intemational, 54,
[Online], 0912.
Brown, J. D. (1988). Understanding research in second language leaming: A teacher's
guide to statistics and research design. Cambridge: Cambridge University Press.
Brown, L. F. (l.brown@ttu.edu). (1999, January 7) (e-mail from Lady Falls Brown).
Use of the Acronym "OWL." E-mail to Liliana Anglada (z31il@ttacs.ttu.edu)
Bmfee, K.A. (1984a). Collaborative leaming and the "conversation of mankind."
College English. 46, 635-652.
Bruffee, K. (1984b). Peer tutoring and "the conversation of mankind." In G. A. Olson
(Ed.), Writing center theory and administration (pp. 3-15). Urbana, IL: NCTE.
Bump, J. (1990). Radical changes in class discussion using networked computers.
Computer and the Humanities, 24, 49-65.
Butler-Pascoe, M. E. (1990). Effective uses of computer technology in the development
of writing skills of students emolled in a college-level Enghsh as a second
language program. Dissertation Abstracts Intemational, 51, [Online], 2295.
Byrd, D. R. (1994). Peer editing: Common concems and applications in the foreign
language classroom. Die Unterrichtspraxis, 27(1), 119-123.
Caissie, R. A. (1997). Enghsh verb phrase grammar prototypes for speakers of other
languages: A cognitive approach to facilitate second language English
composition. Dissertation Abstracts Intemational, 58, [Online], 3112.
223
Canagarajah, A. S. (1993). Comments on Ann Raimes's "Out of the woods: Emerging
traditions in the teaching of writing." Up the garden path: Second language
writing approaches, local knowledge, and pluralism. The author responds....
TESOL Ouarterlv. 27 301-310.
Carino, P. (1992). What do we talk about when we talk about our metaphors: A cultural
critiqueofclinic, lab, anticenter. The Writing Center Joumal. 13(1). 31-42.
Carino, P. (1995a). Early writing centers: Towards a history. The Writing Center
Joumal .15(2) 103-115.
Carino, P. (1995b). Theorizing the writing center: An uneasy task. Dialogue: A Joumal
for Writing Specialists. 2(1). 23-37.
Carino, P. (1996). Open admissions and the construction of writing center history: A
tale of three models. The Writing Center Joumal 17(1). 30-48.
Carter-Tod, S. L. (1995). The role of the writing center in the writing practices of L2
students (Individualized Instmction). Dissertation Abstracts Intemational, 56,
[Online], 4262.
Casanave, C. P. (1994). Language development in students'joumals. Joumal of Second
Language Writing, 3, 179-201.
Caulk, N. (1994). Comparing teacher and student responses to written work. TESOL
Ouarteriy, 28. 181-188.
Chastain, K. (1980). Learners' language errors. In K. Chastain (Ed.). Toward a
philosophy of second-language leaming and teaching (pp. 47-66). Boston:
Heinle.
Chen, J. F. Computer generated error feedback and writing process: A link. TESL-EJ, 2.
3, A-1, 1-15. (http//www-writing.berkeley.edu/TESL-EJ/ej07/al.html)
Chitrapu, D. S. (1996). Case studies of non-native graduate students' research writing in
the disciplines (ESL). Dissertation Abstracts Intemational, 57, [Online], 4347.
Chun, D. M. (1994). Using computer networking to facilitate the acquisition of
interactive competence. System, 22, 17-31.
Clarke, M. A., and J. Handscombe (Eds.). (1983). On TESOL '82: Pacific perspectives
on language leaming and teaching. Washington, D.C.: TESOL.
224
Cohen, A. (1987). Student processing of feedback on their compositions. In A. Wenden
& J. Rubin (Eds.), Leamer strategies in language leaming (pp. 57-69). Englewood
Cliffs, NJ: Prentice-Hall.
Cohen, A. (1990). Language leaming: Insights for leamers, teachers, and researchers.
New York: Newbury House/Harper Row.
Cohen, A. (1991). Feedback on writing: The use of verbal report. Studies in Second
Language Acquisition. 13. 133-159.
Cohen, A. D., & Cavalcanti, M. C. (1987). Giving and getting feedback on composition:
A comparison of teacher and student verbal report. Evaluation and Research in
Education. 1(2). 63-73.
Cohen, A. D., & Cavalcanti, M. C. (1990). Feedback on composition: Teacher and
student verbal reports. In B. Kroll (Ed.), Second language writing: Research
Insights for the classroom (pp. 155-177). Cambridge: Cambridge University
Press.
Cohen, J. (1960). Acoefficient of agreement for nominal scales. Educational and
Psychological Measurement, 29, 1, 37-46.
Cohen, J. (1968). Weighted Kappa: Nominal scale agreement with provision for scaled
disagreement or partial credit. Psychological Bulletin ,70, 213-220.
Cohen, M., & Riel, M. (1989). The effect of distant audiences on students' writing.
American Educational Research Joumal, 26, 143-159.
Cohen, M., &, Miyake, N. (1986). A worldwide intercultural network: Exploring
electronic messaging for instruction. Instructional Science, 15. 257-273.
Connor, U., & Asenavage, K. (1994). Peer response groups in ESL writing classes: How
much impact on revision? Joumal of Second Language Writing. 3. 257-276.
Connor, U., & Johns, A. M. (1990). Coherence in Writing. Alexandria, VA: TESOL.
Connor, U., & Kaplan, R. B. (1987). Writing across languages: Analysis of L2 written
text. Reading, MA: Addison-Wesley.
Coogan, D. (1994). Towards a rhetoric of on-line tutoring. Writing Lab Newsletter.
19(1), 3-5.
Coogan, D. (1995a). Electronic writing centers (Electronic mail. Computer-assisted
instruction). Dissertation Abstract Intemational, 56, [Online], 3861.
225
Coogan, D. (1995b). E-mail tutoring, a new way to do new work. Computers and
Composition, 12, 171-181.
Cooper, M., & Self, C. L. (1990). Computer conferences and leaming: Authority,
resistance, and intemally persuasive discourse. College Enghsh, 52, 847-869.
Cresswell, J.W. (1998). Oualitative inquiry and research design: Choosing among five
traditions. Thousand Oaks, CA: Sage Publications.
Gumming, A. (1990a). Expertise in evaluating second language compositions.
Language Testing, 7, 31-51.
Gumming, A. (1990b). Writing expertise and second language proficiency. Language
Leaming, 9, 81-141.
Cummins, J. (1986). Cultures in contact: Using classroom microcomputers for cultural
interchange and reinforcement. TESL Canada Joumal/Revue TESL du Canada,
3(2), 13-31.
Cummins, J., & Sayers, D. (1990). Education 2001: Leaming networks and educational
reform. In C. Faltis & R. A. DeVillars (Eds.) [Special edition: Computers and the
Schools] Language Minority Students and Computers, 7(1-2), 1-29.
Cummins, J., & Sayers, D. (1995). Brave new schools: Challenging cultural illiteracy
through global leaming networks. New York: St. Martin's Press.
Daly, J. (1985). Writing apprehension. In M. Rose (Ed.), When a writer can't write:
Studies in writer's block and other composing process problems (pp. 43-82).
New York: The Guilford Press.
Davies, N. F., & Omberg, M. (1987). Peer group teaching and the composition class.
System. 15,313-323.
Dessner, L. E. (1991). English-as-a-second language college writers' revision responses
to teacher-written comments (Doctoral dissertation. University of Pennsylvania,
1991). Dissertation Abstracts Intemational, 52. [Online], 0827.
Devenney, R. (1989). How ESL teachers and peers evaluate and respond to student
writing. RELC Joumal. 20(1). 77-90.
Dheram, P. K. (1995). Feedback as a two-bullock cart: A case study of teaching writing.
ELT Joumal. 49. 161-168.
DiMatteo, A. (1990). Under erasure: A theory for interactive writing in real time.
Computers and Composition. 7, 70-83.
226
Dossin, M. (1996). ESL quandary. Writing Lab Newsletter, 20(9). 15-16.
Dubrovsky, V. J., Kiesler, S., & Sethna, B.N. (1991). The equalization phenomenon:
Status effects in computer-mediated and face-to-face decision-making groups.
Human-Computer Interaction. 6. 119-146.
Dunkel, P. (Ed.). (1991). Computer-assisted language leaming and testing: Research
issues and practice. New York: Newbury House.
Ede, L. (1989). Writing as a Social Process: A theoretical foundation for writing
Centers? The Writing Center Joumal 9. 3-15.
Eldred, J. C , & Hawisher, G. E. (1995). Researching electronic networks. Written
Communication. 12. 330-359.
Elhs, R. (1986). Understanding second language acquisition. Oxford: Oxford University
Press.
Ellis, R. (1990). Instructed second language acquisition: Leaming in the classroom.
Oxford: Basil Blackwell Ltd.
Ellis, R. (1994). The study of second language acquisition. Oxford: Oxford University
Press.
El-Shafie, A. S. (1990). English writing development of Arab twelfth-grade students:
Case studies of six EFL writers. Dissertation Abstracts Intemational. 51.
[Online], 3653.
Engber, C. A. (1995). The relationship of lexical proficiency to the quality of ESL
compositions. Joumal of Second Language Writing. 4. 139-155.
Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic
inquiry: A guide to methods. Newbury Park, CA: Sage Publications.
Faigley, L. (1990). Subverting the electronic workbook: Teaching writing usmg
networked computers. In D. A. Daiker, & M. Morenberg (Eds.), The writing
teacher as researcher: Essays in the theory and practice of class-based research
(pp. 290-311). Portsmouth, NH: Boynton/Cook Pubhshers Heinemann.
Faigley, L., & Witte, S. (1981). Analyzing revision. College Composition and
Communication, 32, 400-407.
Fathman, A. K., & Whalley, E. (1990). Teacher response to student writing: Focus on
form versus content. In B. Kroll (Ed.), Second language writing: Research
Insights for the classroom (pp. 178-90). Cambridge: Cambridge University Press.
227
Ferrara, K., Brunner, H. & Whittemore, G. (1991). Interactive written discourse as an
emergent register. Written Communication, 8. 8-34.
Ferris, D. (1995). Student reactions to teacher response in multi-draft composition
classrooms. TESOL Ouarteriy, 29, 33-53.
Ferris, D. (1997). The influence of teacher commentary on student revision, TESOL
Ouarteriy. 31. 315-339.
Ferris, D., Pezone, S., Tade, C. & Tinti, S. (1997). Teacher commentary on student writing:
Descriptions and implications. Joumal of Second Language Writing, 6, 155-182.
Ferris, R. M., & R. L. Politzer. (1981). Effects of early and delayed second language
acquisition: English composition skills of Spanish-speaking junior high students.
TESOL Ouarteriy. 15. 263-274.
Fleiss, J. L. (1973). The measurement and control of misclassification. Statistical
methods for rates and proportions (pp. 140-154). New York: John Wiley and
Sons.
Flower, L., & J. R. Hayes. (1981). A cognitive process theory of writing. College
Composition and Communication, 32, 365-387.
Frantzen, D. (1995). The effects of grammar supplementation on written accuracy in an
intermediate Spanish content course. The Modem Language Joumal 79, 329-344.
Freedman, A. (1995). The what, where, when, why, and how of classroom genres. In J.
Petragha (Ed.), Reconceiving writing, rethinking writing instruction (pp. 121144). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
Freedman, S. W. (Ed.). (1985). The acquisition of written language: Response and
revision. Norwood, NJ: Ablex Publishing Corporation.
Gass, S., & Madden, C. (Eds.) (1985). Input in second language acquisition. Rowley,
MA Newbury House.
Gerrard, L. (1989). Computers and basic writers: A critical view. In G. E. Hawisher &
C. L. Selfe (Eds.). Critical perspectives on computers and composhion
instruction (pp. 94-108). New York: Teachers College Press.
Ghaleb, M. L. N. (1993). Computer networking in a university freshman ESL writing
class: A descriptive study of the quantity and quality of writing in networking and
traditional writing classes. Dissertation Abstracts Intemational, 54, [Online],
2865.
228
Gillam, A. (1991). Writing center ecology: A Bakhtinian perspective. The Writing
Center Joumal, 11(2), 3-11.
Glaser B. G., & Strauss A. L. (1967). The discovery of grounded theory: Strategies for
quantitative research: Chicago: Aldine.
Goldstein, L. M., & Conrad, S. M. (1990). Student input and negotiation of meaning in
ESL writing conferences. TESOL Ouarteriy, 24. 443-460.
Goodwin, A. A., Hamrick, J., & Stewart, T. C. (1993). Instructional delivery via
electronic mail. TESOL Joumal. 3(1). 2-27.
Gordon, C. J. (1990). A study of students' text stmcture revisions. English Ouarteriy,
23_(l-2), 7-30.
Goring Kepner, C. (1991). An experiment in the relationship of types of written
feedback to the development of second-language writing skills. The Modem
Language Joumal, 75. 305-313.
Greenia, G. D. (1992). Computers and teaching composition in a foreign language.
Foreign Language Annals. 25, 33-46.
Griffith, T. L., & Northcraft, G. B. (1994). Distinguishing between the forest and the
trees: Media, features, and methodology in electronic communication research.
Organization Science: A Joumal of the Institute of Management Sciences, 5. 272286.
Grubbs, K. (1994). Some questions about the politics of on-hne tutoring in electronic
writing centers. The Writing Lab Newsletter, 19(2), 7,12.
Gruber, S. (1995). Re: Ways we contribute: Students, instructors, and pedagogies in the
computer-mediated writing classroom. Computers and Composition. 12. 61-78.
Gungle, B., & Taylor, V. (1989). Writing apprehension and second language writers. In
Johnson, D. M., & Roen D. H. (Eds.), Riclmess in writing: Empowering ESL
students, (pp. 335-248). New York: Longman.
Hall, C. (1995). Comments on Joy Reid's "Responding to ESL students' text: The myths
of appropriation." TESOL Ouarterlv, 29. 159-166.
Handa, C. (Ed.). (1990). Computers and community: Teaching composition in the
twenty-first century. Portsmouth, NH: Boynton/Cook.
Harasim, L. M. (Ed.). (1993). Global networks: Computers and intemational
communication. Cambridge, MA: Massachusetts Institute of Technology Press.
229
Harris, M. (1997). Letters: Open Admissions and the construction of writing center
history: A tale of three models. The Writing Center Joumal. 17(2), 134-140.
Harris, M., & Pemberton, M. (1995) On-line writing labs (OWLs): A Taxonomy of
options and issues. Computers and Composition. 12. 145-159.
Harris, M., & Silva, T. (1993). Tutoring ESL students: Issues and options. College
Composition and Communication. 44. 525-537.
Hartman, K., Neuwirth, C. M., Kiesler, S., Sproull, L., Cochran, C, Palmquist, M. E., &
Zubrow, D. (1991). Pattems of social interaction and leaming to write. Written
Communication. 8. 79-113.
Hassan, R. H. (1995). The effect of an ESL workstation on ESL students' writing
(Doctoral dissertation. University of Illinois at Urbana-Champaign, 1995).
Dissertation Abstracts Intemational. 56. [Online], 0905.
Hatch, E. (1978). Discourse analysis and second language acquisition. In Hatch, E.
(Ed.), Second Language Acquisition. Rowley, MA: Newbury House.
Hatch, E. (1992). Discourse and language education. New York: Cambridge University
Press.
Hatch, E., & Lazaraton, A. (1991). The research manual: Design and statistics for
applied linguistics. Boston, Massachusetts: Heinle & Heinle Publishers.
Hawisher, G. E. (1992). Electronic meetings of the minds: Research, electronic
conferences, and composition studies. In G. E. Hawisher & P. LeBlanc (Eds.),
Re-imagining computers and composition: Teaching and research in the virtual
age (pp. 81-101). Porstmouth, NH: Boynton/Cook Heinemann.
Hawisher, G. E., & Selfe, C. L. (1991). The rhetoric of technology and the electronic
writing class. College Composition and Commimication, 42(1), 55-65.
Healy Beauvois, M. (1992). Computer-assisted classroom discussion in the foreign
language classroom: Conversation in slow motion. Foreign Language Annals. 25.
455-464.
Hedgcock, J., & Lefkowitz, N. (1994). Feedback on feedback: Assessing leamer
receptivity to teacher response in L2 composing. Joumal of Second Language
Writing. 3, 141-163.
Hedgcock, J., & Lefkowitz, N. (1996). Some input on input: Two analyses of student
response to expert feedback. The Modem Language Joumal, 80(3), 287-308.
230
Hendrickson, J. M. (1978). Error correction in foreign language teaching: Recent theory,
research, and practice. The Modem Language Joumal, 62, 387-398.
Hendrickson, J. M. (1980). The treatment of error in written work. The Modem
Language Joumal. 64. 216-221.
Henley-Jordan, J, & Maid, B. (1995a). MOOving along the information superhighway:
Writing centers in cyberspace. The Writing Lab Newsletter. 19(5), 1-6.
Henley-Jordan, J, & Maid, B. (1995b). Tutoring in cyberspace: Student impact and
college/university collaboration. Computers and Composition, 12, 211-218.
Herrmann, F. (1990). Instrumental and agentive uses of the computer: Their role in
leaming French as a foreign language. Dissertation Abstracts Intemational, 51.
[Online], 0728.
Hewins, C. P. (1986). Writing in a foreign language: Motivation and the process
approach. Foreign Language Annals. 19. 219-223.
Hill, C. A., & Resnick, L. (1995). Creating opportunities for apprenticeship in writing.
In J. Petragha (Ed.), Reconceiving writing, rethinking writing instmction (pp.
145-158). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Hillocks, G. (1982). The interaction of instruction, teacher comment, and revision in
teaching the composing process. Research in the Teaching of English. 16, 261
278.
Hoffmann, A. (1995). What have we learnt about reformulation as a way to respond to
L2 writing? In M. L. Tickoo (Ed.), Reading and writing: Theory into practice
(pp. 1-19). Singapore: Sherson Pubhshing House Pte. Ltd.
Holdstein, D. H., & Selfe, C. L. (Eds.). (1990). Computers and writing: Theory.
research, practice. New York: The Modem Language Association of America.
Holt, P.O., & Williams, N. (1992). Computers and writing: State of the Art. Oxford:
Intellect Books.
Horowitz, D. (1986a). Essay examination prompts and the teaching of academic writing.
Enghsh for Specific Purposes, 5, 107-120.
Horowitz, D. (1986b). Process, not product: Less than meets the eye. TESOL Ouarteriy.
20, 141-144.
Horowitz, D. (1986c). What Professors actually require: Academic tasks for the ESL
classroom. TESOL Ouarteriy, 20, 445-462.
231
Huntley, H. S. (1992). Feedback strategies in intermediate and advanced second language
composition: A discussion of the effects of error correction, peer review, and
student-teacher conferences on student writing and performance. Viewpoints, (116). (ERIC Document Reproduction Service No ED 355 809).
Hyland. F. (1988). The impact of teacher written feedback on individual writers.
Joumal of Second Language Writing, 7. 255-286.
Hyland, K. (1993). ESL computer writers: What can we do to help? System 21, 21-30.
Ihde, T. W. (1994). Feedback in L2 writing. Paper presented at the Annual Conference
of the American Association for Applied Linguistics (1-6). (ERIC Document
Reproduction Service No ED 369 287).
Jacobs, G. M., Curtis, A., Braine, G., & Huang S-Y. (1998). Feedback on student
writing: Taking the middle path. Joumal of Second Language Writing. 7. 3, 307317.
Jacobs, H. L., Zingraf, S.A., Wormuth, D. R., Hartfield, V. F., & Hughey, J.B. (1981).
Testing ESL composition: A practical approach. Rowley, MA: Newbury House.
Johnson, D. M., & RoenD. H. (Eds.). (1989). Richness in writing: Empowering ESL
students. New York: Longman.
Jung, H. S. (1993). The revision processes of three experienced EFL writers (Korean).
Dissertation Abstracts Intemational, 54, [Onhne], 4018.
Kaplan, R. (1966). Cultural thought pattems in intercultural education. Language
Leaming, 16, 1-20.
Kaplan, R. (1967). Contrastive rhetoric and the teaching of composition. TESOL
Ouarteriy, 1, 10-16.
Kaufer, D. S., & Carley, K. (1994). Some concepts and axioms about communication:
Proximate and at a distance. Written Communication, 11, 8-41.
Keh, C. L. (1990). Feedback in the writing process: A model and methods for
implementation. ELT Joumal, 44, 294-304.
Kelm, O. R. (1992). The use of synchronous computer networks in second language
instruction: A preliminary report. Foreign Language Annals, 25, 441-454.
232
Kemp, F. (1993). The origins of ENFI, network theory, and computer-based
collaborative writing instruction at the University of Texas. In B. C. Bruce, J. K.
Peyton, & T. Batson (Eds.), Network-based classrooms: Promises and realities
(pp. 161-180). New York: Cambridge University Press.
Kemp, F. (1995). Writing dialogically: Bold lessons from electronic text. In J. Petragha
(Ed.), Reconceiving writing, rethinking writing instruction (pp. 179-194).
Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Kennedy, B. L. (1993). Non-native speakers as students in first-year composition classes
with native speakers: How can writing tutors help? The Writing Center Joumal,
13(2), 27-38.
Kepner, C. G. (1991). An experiment in the relationship of types of written feedback to
the development of second language writing skills. The Modem Language
Joumal, 75, 305-313.
Kem, R. (1995). Restmcturing classroom interaction with networked computers: Effects
on quantity and quality of language production. The Modem Language Joumal,
79, 457-476.
Kimball, S. (1997). Cybertext/cyberspeech: Writing centers and online magic. The
Writing Center Journal. 18 (1), 30-49.
Kinkead, J. (1987). Staffroom interchange: Computer conversations: E-mail and writing
instruction, student-to-student correspondence, writmg center correspondence,
and student-teacher correspondence. College Composition and Communication,
38,337-341.
Kinkead, J. (1988). The electronic writing tutor. The Writing Lab Newsletter, 12(4), 45.
Kitchin, D. A. (1991). Case study of ESL community college students using computerbased writing tools in a composhion course. Dissertation Abstracts Intemational,
52, [Online], 1240.
Knoblauch, C. H., & Brannon, L. (1981). Teacher commentary on student writing: The
state of the art. Freshman Enghsh News, 10(2), 1-4.
Knoblauch, C. H., & Brannon, L. (1984). Rhetorical tradhions and the teaching of
writing. Upper Montclair: NJ: Boynton/Cook Pubhshers, Inc.
Kobayashi, T. (1992). Native and non-native reactions to ESL compositions. TESOL
Ouarteriy. 26. 81-112.
233
Krashen, S. D. (1981). Second language acquisition and second language leaming.
Oxford: Pergamon Press.
Krashen, S. D. (1982). Principles and practice in second language acquisition. New
York: Pergamon Press.
Krashen, S. D. (1984). Writing, research, theory, and applications. New York:
Pergamon Institute of English.
Krashen, S. D. (1985). The Input Hypothesis. London: Longman.
Krashen, S. D., & Terrel, T. (1983). The natural approach: Language acquisition in the
classroom. Oxford: Pergamon Press.
Kremers, M. (1990). Sharing authority on a synchronous network: The case for riding the
beast. Computers and Composition, 7, 33-44.
Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Beverly
Hills: Sage Pubhcations.
Kroll, B. (Ed.). (1990). Second language writing: Research insights for the classroom.
New York: Cambridge University Press.
Kroll, B. (1990). Introduction. Second language writing: Research insights for the
classroom (pp. 1-5). Cambridge: Cambridge University Press.
Kroll, B. (1991). Teaching writing in the ESL context. In M. Celce-Murcia (Ed.),
Teaching English as a second or foreign language (2nd ed.), (pp. 245-263). New
York: Newbury House/HarperColhns.
Kroll, B., & Reid, J. M. (1994). Guidelines for designing writing prompts: Clarifications,
caveats, and cautions. Joumal of Second Language Writing, 3, 231-255.
Kuhn, T. S. (1970). The structure of scientific revolutions (2nd ed.). Chicago:
University of Chicago Press.
Lalande, J. F., II. (1982). Reducing composition errors: An experiment. The Modem
Language Joumal. 66, 140-149.
Lam, F. S., & Pennington, M.C. (1995). The computer vs. the pen: A comparative study
of word processing in a Hong Kong secondary classroom. Computer Assisted
Language Leaming, 8(1), 75-92.
Landis R, & Koch G. (1977). The measurement of observer agreement for categorical
data. Biometrics, 33, 159-174.
234
Larsen-Freeman, D. (1991). Second language acquisition research: Staking out the
territory. TESOL Ouarteriy. 25. 315-350.
Lauer, J. M., & Asher, J. W. (1988). Composition research: Empirical designs. New
York: Oxford University Press.
Lauzon, A. C. (1992). Integrating computer-based instruction with computer
conferencing: An evaluation of a model for designing online education. The
American Joumal of Distance Education. 6(2). 32-46.
Lee, Y-C. (1993). Revision breakdowns in academic writing of Chinese graduate-level
ESL students. Dissertation Abstracts Intemational, 53, [Online], 0429.
Leki, I. (1990). Coaching from the margins: Issues in written response. Second
language writing: Research insights for the classroom (pp. 57-68). Cambridge:
Cambridge University Press.
Leki, I. (1991a). The preferences of ESL students for error correction in college-level
writing classes. Foreign Language Aimals, 24, 203-218.
Leki, I. (1991b). Twenty-five years of contrastive rhetoric: Text analysis and writing
pedagogies. TESOL Ouarteriy, 2(1), 123-139.
Leki, I. (1992). Understanding ESL writers: A guide for teachers. Portsmouth, New
Hampshire: Boynton/Cook Publishers.
Leki, I., & Carson, J. G. (1994). Students' perceptions of EAP writing instruction and
writing needs across the disciplines. TESOL Ouarteriy, 28(1), 81-101.
Liou, H. C. (1991). Development of an English grammar checker: A progress report.
CALICO Joumal. 9(1), 57-70.
Liou, H. C. (1993). Investigation of using text-critiquing programs in a process-oriented
writing class. CALICO Joumal. 10(4). 17-38.
Lockhart, C , & Ng, P. (1995). Analyzing talk in ESL peer response groups: Stances,
functions, and content. Language Leaming. 45, 605-655.
Long, M. H. (1981). Input, interaction and second language acquisition. In Wiiutz, H.
(Ed.), Annals of the New York Academy of Sciences: Vol. 379. Native language
and foreign language acquisition (pp. 259-278). New York: The New York
Academy of Sciences.
235
Long, M. H. (1983). Native speaker/non-native speaker conversation in the second
language classroom. In M. A. Clarke & J. Handscombe (Eds.), On TESOL '82:
Pacific perspectives on language leaming and teaching (pp. 207-225).
Washington, D.C.: TESOL.
Long, M. H. (1985). Input and language acquishion theory. In S. Gass & C. Madden
(Eds.), Input in second language acquisition (pp. 377-403). Rowley, MA:
Newbury House.
Long, M. H. (1996). The role of linguistic environment in second language acquisition.
In W. C. Ritchie & T. K. Bhatia (Eds.), Second Language acquishion: Vol. 2.
Handbook of language acquisition (pp. 413-468). San Diego: Academic press.
Lunsford, A. (1991). Collaboration, control, and the idea of the writing center. The
Writing Center Joumal, 12(1), 3-10.
Lyons, G. (1992). Validating cultural difference in the writing center. The Writing
Center Joumal, 12(2), 145-158.
Mabrito, M. (1991). Electronic mail as a vehicle for peer response: Conversations of
high- and low-apprehensive vmters. Written Communication, 8, 509-532.
Mabrito, M. (1992). Real-time computer network collaboration: Case studies of business
writing students. The Joumal of Business and Technical Communication, 16, 316336.
Malicka, A. (1996). Tasks, interaction, affect, and the writer's development: A
descriptive study of an ESL composition class. Dissertation Abstracts
Intemational, 57, [Online], 1061.
Mangelsdorf, K. (1992). Peer reviews in the ESL composition classroom: What do the
students think? ELT Joumal. 46. 274-284.
Marshall, C , & G. Rossman. (1995). Designing qualitative research (2nd ed.).
Thousand Oaks, CA: Sage Publications.
Marx, M. S. (1990). Distant writers, distant critics, and close readings: Linking
composition classes through a peer-critiquing network. Computers and
Composition. 8. 23-39.
McComb, M. (1993). Augmenting a group discussion course with computermediated
communication in a small college setting. Interpersonal Computing and
Technology: An Electronic Joumal for the 21st Century 1(3). MCCOMB
IPCTV1N3 on <LISTSERV@GUVM>.
236
McKay, S. (1994). Developing ESL writing materials. System, 22, 195-203.
Mehdi, S. N. (1994). Word processing and the EFL writing classroom: A case study of
Arabic speakers. Dissertation Abstracts Intemational. 55. [Online], 0889.
Mendonca, C. 0., & Johnson, K. E. (1994). Peer review negotiations: Revision activities
in ESL writing instmction. TESOL Ouarteriy. 28. 745-769.
Miles, M.B., & Huberman, A. M. (1994). Oualitative data analysis: A sourcebook of
new methods (2nd ed.) Thousand Oaks, CA: Sage Publications.
Miller, G., Galanter, E., & Pribram, K. (1960). Plans and the stmcture of behavior. New
York: Holt Rinehart & Winston.
Mooko, T. (1996). An investigation into the impact of guided peer feedback and guided
self-assessment on the quality of compositions written by secondary school
students in Botswana. Dissertation Abstracts Intemational, 58, [Onhne], 1155.
Mukherjee, K. (1996). Perspectives from an intermediate ESOL classroom: A case
study of a child acquiring second language writing skills. Dissertation Abstracts
Intemational, 57. [Online], 4250.
Murau, A. M. (1993). Shared writing: Students' perceptions and attitudes of peer review.
In J. Gladstein & R. E. Silver (Eds.), Working Papers in Educational Linguistics
9(2), 71-796. (ERIC Document Reproduction Service No. ED 367 133).
Murray, D. E. (1988). Computer-mediated communication: Implications for ESP.
English for Specific Purposes, 7, 3-18.
Myers, G. (1986). Reality, consensus, and reform in the rhetoric of composition
teaching. College English. 48, 154-174.
Myers, M. J. (1995). Problemes de semiotique en L2: Communication interculturelle et
interaction. The Canadian Modem Language Review, 51. 678-687.
Nelson, G. L., & Murphy, J. M. (1992). An L2 writing group: Task and social
dimensions. Joumal of Second Language Writing, 1, 171-193.
Nelson, G. L., & Murphy, J. M. (1993). Peer response groups: Do L2 writers use peer
comments in revising their drafts? TESOL Ouarteriy, 27. 135-141.
Newell, A. (1980). Reasoning, problem solving, and decision processes: The
problemspace as a fundamental category. In R. S. Nickerson (Ed.), Attention and
performance VIII. Hillsdale, NJ: Lawrence Erlbaum.
237
Newell, A., & Simon, H. (1972). Human problem solving. Englewood Cliffs, NJ:
Prentice-Hall.
North, S. (1984). The idea of a writing center. College Enghsh, 46. 433-446.
North, S. (1987). The making of knowledge in composition: Portrait of an emerging
field. Upper Montclair, NJ: Boynton/Cook, 1987.
Nunan, D. (1991). Methods in second language classroom-oriented research: A critical
review. Studies in Second Language Acquisition, 13, 249-274.
Obah, T. Y. (1993). Leaming from others in the ESL writing class. English Ouarteriy
25(1), 8-13.
Ohva, M., & Pollastrini, Y. (1995). Intemet resources and second language acquisition:
An evaluation of virtual immersion. Foreign Language Annals, 28, 551-563.
Olson, G. A. (Ed.). (1984). Writing center theory and administration. Urbana, IL:
NCTE.
Palmquist, M. E. (1993). Network-supported interaction in two writing classrooms.
Computers and Composition, 10. 26-57.
Paramskas, D. (1993). Computer-assisted language leaming (CALL): Increasingly
integrated into an ever more electronic world. Canadian Modem Language
Review, 50, 124-143.
Patthey-Chavez, G. G., & D. R. Ferris. (1997). Writing conferences and the weaving of
multi-voiced texts in college composition. Research in the Teaching of English
31,51-90.
Patton, M. Q. (1990). Oualitative evaluation and research methods (2nd ed.). Newbury
Park, CA: Sage Pubhcations.
Pennington, M. (1996). The computer and the non-native writer: A natural partnership.
Cresskill, NJ: Hampton Press, Inc.
Petit, A. (1997). The writing center as "purified space": Competing discourses and the
dangers of definition. The Writing Center Joumal, 17(2). 111-122.
Petragha, J. (Ed.). (1995). Reconceiving writing, rethinking writing instruction.
Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
238
Peyton, J. K., Jones, C , Vincent, A., & Greenblatt, L. (1994). Implementing writing
workshop with ESOL students: Visions and realities. TESOL Ouarteriy, 28, 469487.
Phinney, M. (1991). Computer-assisted writing and writing apprehension in ESL
students. In P. Dunkel (Ed.), Computer-assisted language leaming and testing:
Research issues and practice (pp. 189-204). New York: Newbury House.
Phinney, M., & Khouri, S. (1993). Computers, revision, and ESL writers: The role of
experience. Joumal of Second Language Writing, 2, 257-277.
Piper, A. (1995). Reformulation, evaluation, and revision: Do they help academic
writers? In M. L. Tickoo (Ed.), Reading and writing: Theory into practice (pp.
20-38). Singapore: Sherson Publishing House Pte. Ltd.
Polio, C , Fleck, C , Leder, N. (1998). "If I only had more time:" ESL learners' changes
in linguistic accuracy on essay revisions. Joumal of Second Language Writing, 7,
43-68.
Porte, G. (1996). When writing fails: How academic context and past leaming
experience shape revision. System, 24, 107-116.
Powers, J. K. (1993). Rethinking writing center conferencing strategies for the ESL
writer. The Writing Center Joumal, 13(2), 39-47.
Radecki, P. M, & Swales, J. M. (1988). ESL student reaction to written comments on
their written work. System. 16, 355-365.
Rahman, M. A. (1990). Some effects of computers on ESL student writing. Dissertation
Abstracts Intemational. 51, [Online], 2719.
Raimes, A. (1983). Tradition and revolution in ESL teaching. TESOL Ouarterlv. 17(4).
535-552.
Raimes, A. (1985). What unskilled ESL students do as they write: A classroom study of
composing. TESOL Ouarteriy. 19. 229-259.
Raimes, A. (1987). Language proficiency, writing abihty, and composing strategies: A
study of ESL college student writers. Language Leaming. 37. 439-467.
Raimes, A. (1991). Out of the woods: Emerging traditions in the teaching of writing.
TESOL Ouarteriy. 25. 407-430.
239
Raimes, A. (1995). Ten steps in planning a writing course and training teachers of
writing. In M. L. Tickoo (Ed.), Reading and writing: Theory into practice (pp.
514-527). Singapore: Sherson Publishing House Pte. Ltd.
Reid, J. (1989). Enghsh as a second language composition in Higher education: The
expectations of the academic audience. In D. M. Johnson, & D. H. Roen (Eds.),
Richness in Writing: Empowering ESL students (pp. 220-234). New York:
Longman.
Reid, J. (1992). The writing-reading connection in the ESL composition classroom.
Joumal of Intensive English Studies. 6. 27-50.
Reid, J. (1993). Teaching ESL writing. Englewood Chffs. NJ: Regents/Prentice Hall.
Reid, J. (1994). Responding to ESL students' texts: The myths of appropriation. TESOL
Ouarteriy. 28, 273-292.
Renie, D., & Chanier, T. (1995). Collaboration and computer-assisted acquisition of a
second language. Computer Assisted Language Leaming. 8. 3-29.
Riel, M. M. (1983). Education and ecstasy: Computer chronicles of students writing
together. The Ouarteriy Newsletter of the Laboratory of Comparative Human
Cognition. 5(3). 59-67.
Riel, M. M. (1985). The computer chronicles newswire: A functional leaming
environment for acquiring literacy skills. Joumal of Educational Computing
Research. 1. 317-337.
Riel, M. M., & Levin, J. A. (1990). Building electronic communities: Success and failure
in computer networking. Instructional Science. 19. 145-169.
Riley, S. M. (1995). Peer responses in an ESL writing class: Student interaction and
subsequent draft revision. Dissertation Abstracts Intemational. 56, [Online],
3031.
Ritchie, W. C , & Bhatia, T. K. (Eds.). (1996). Annals of the New York Academy of
Sciences: Vol. 379. Native language and foreign language acquisition. New
York: The New York Academy of Sciences.
Robb, T., Ross, S., & Shortreed, I. (1986). Sahence of feedback on error and its effect
on EFL writing quality. TESOL Ouarteriy, 20, 83-95.
Roen, D. H. (1989). Developing effective assignments for second language writers. In
D. M. Johnson & D. H. Roen (Eds.), Richness in writing: Empowering ESL
students (pp. 193-207). New York: Longman.
240
Romano, S. (1993). The egahtarian narrative: Whose story? Which yardstick?
Computers and Composition. 10(3). 5-28.
Rose, M. (Ed.). (1985). When a writer can't write: Studies in writer's block and other
composing process problems. New York: The Guilford Press.
Santos, T. (1992). Ideology in composition: LI and ESL. Joumal of Second Language
Writing. 1. 1-15.
Sayers, D. (1993). Distance team teaching and computer leaming networks. TESOL
Joumal. 3(1). 19-23.
Schachter, J. (1986a). In search of systematicity in interlanguage production. Studies in
Second Language Acquisition, 8, 119-34.
Schachter, J. (1986b). Three approaches to the study of input. Language Leaming, 36,
211-225.
Scott, V. M. (1995). Rethinking foreign language writing. Boston, MA: Heinle &
Heinle Publishers.
Selfe, C. L, & Hilligoss, S. (Eds.). (1994). Literacy and computers: The complications
of teaching and leaming with technology. New York: Modem Language
Association of America.
Selfe, C. L., & Meyer, P. (1991). Testing claims for on-line conferences. Written
Communication, 8, 163-192.
Semke, H. D. (1984). Effects of the red pen. Foreign Language Annals, 17. 195-202.
Severino, C. (1993). The sociopolitical implications of response to second language and
second dialect writing. Joumal of Second Language Writing, 2, 181-201.
Sheppard, K. (1992). Two feedback types: Do they make a difference? RELC Joumal.
23, 103-110.
Siegel, J., Dubrovsky, V., Kiesler, S., & McGuire, T. W. (1986). Group processes in
computer-mediated communication. Organizational Behavior and Human
Decision Processes, 37, 157-187.
Silva, T. (1990). Second language composhion instruction: Developments, issues, and
directions in ESL. Second language writing: Research insights for the classroom
(pp. 11-23). Cambridge: Cambridge University Press.
241
Silva, T. (1993). Toward an understanding of the distinct nature of L2 writing: The ESL
research and hs implications. TESOL Ouarteriy, 27, 657-677.
Silva. T. (1994). Resources for prospective teachers of ESL writing. English in Texas,
25(3), 8-10.
Silva, T. (1997). On the ethical treatment of ESL writers. TESOL Ouarteriy, 31, 359363.
Singer, S. A. (1994). Muhiple case study of freshman writing students on a networked
writing environment. Dissertation Abstracts Intemational, 56, [Online], 0079.
Skubikowski, K., & Elder, J. (1990). Computers and the social contexts of writing. In C.
Handa (Ed.), Computers and community: Teaching composition in the twentyfirst century (pp. 89-105). Portsmouth, NH: Boynton/Cook Pubhshers.
Smith, A. (1993). Revising process and written product: A study of basic and skilled LI
English and ESL writers using computers. Dissertation Abstracts Intemational,
54-11 A. [Online], 4078.
Smith, K. L. (1990). Collaborative and interactive writing for increasing communication
skills. Hispania, 73, 77-87.
Smith-Hobson, S. E. (1993). Teaching writing as a social activity. A five-part
integrative approach (FPIA): Using wordprocessing, interactive computing-real
time writer (RTW), writing and literacy theories, and peer tutoring. Dissertation
Abstracts Intemational, 54, [Online], 4350.
Soh, B.-L., & Soon, Y.-P. (1991). English by e-mail: Creating a global classroom via
the medium of computer technology. ELT Joumal, 45, 287-292.
Sperling, M. (1994). Constructing the perspective of teacher-as-reader: A framework for
studying response to student writing. Research in the Teaching of English 28.
175-207.
Spooner, M. (1994). A dialogue on OWLing in the writing lab: Some thoughts about
on-line writing labs. The Writing Lab Newsletter, 18(6), 6-8.
Straub, R. (1997). Students' reactions to teacher Comments: An exploratory study.
Research in the Teaching of Enghsh, 31, 91-119.
Sullivan, N., & Pratt, E. (1996). A comparative study of two ESL writing environments:
A computer-assisted classroom and a traditional oral classroom. System, 29. 491501.
242
Susser, B. (1993). Networks and project work: Ahemative pedagogies for writing with
computers. Computers and Composition, 10. 63-89.
Susser, B. (1994). Process approaches in ESL/EFL writing instruction. Joumal of
Second Language Writing. 3. 31-47.
Swain, M. (1985). Communicative competence: Some roles of comprehensible input
and comprehensible output in its development. In S. Gass & C. Madden (Eds.),
Input in second language acquisition (pp. 235-253). Rowley, MA.: Newbury
House.
Takayoshi, P. (1994). Building new networks from the old: Women's experiences with
electronic communication. Computers and Composition. 11. 21-36.
Tarone, E. E., Gass, S. M., & Cohen, A. D. (Eds.). (1994). Research methodology in
second-language acquisition. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Texas Tech University Writing Center, http://english.ttu.edu/uwc/owl/FAQ.html
(October 7, 1998)
Texas Tech University Writing Center, http://english.ttu.edu/uwc/owl/mission.html
(October 7, 1998).
Thonus, T. (1993). Tutors as teachers: Assisting ESL/EFL students in the writing center.
The Writing Center Joumal, 13. 13-25.
Tickoo, M. L. (Ed.). (1995). Reading and writing: Theory into practice. Singapore:
Sherson Publishing House Pte. Ltd.
Tillyer, D. A. (1993). World peace and natural writing through e-mail. Collegiate
Microcomputer, 2, 67-69.
Tribble, C. (1996). Writing. Oxford: Oxford University Press.
Trimbur, J. (1989). Consensus and difference in collaborative learning. College
English. 51,602-616.
Tmscott, J. (1996). Review article: The case against grammar correction in L2 writing
Classes. Language Leaming. 46, 327-369.
Vandergrift, L. (1986). Second language writing and correction: Toward an improved
model for composition correction. The Canadian Modem Language Review. 42.
658-667.
243
Ventola, E., & Mauranen, A. (Eds.). (1996). Academic writing: Intercultural and textual
issues. Amsterdam: John Benjamins Publishing Co.
Victori Blaya, M. (1995). EFL writing knowledge and strategies: An integrative study.
Dissertation Abstracts Intemational. 58. [Online], 0023.
Walz, J. C. (1982). Error correction techniques for the FL classroom. Washington:
CAL.
Wamock, T. & Wamock, J. (1984). Liberatory writing centers: Restoring authority to
writers. In G. A. Olson (Ed), Writing center theory and administration (pp. 1623). Urbana, IL: NCTE.
Warschauer, M. (1994). E-mail for English teaching: Bringing the Intemet and
computer leaming networks into the language classroom. Alexandria, VA:
TESOL.
Warschauer, M. (Ed.). (1995). Virtual connections: Online activities and 99 projects for
networking language leamers. Honolulu, Hawai'i: University of Hawai'i Press.
Warschauer, M. (1996). Comparing face-to-face and electronic discussion in the second
language classroom. CALICO Journal, 13(2-3), 7-26.
Warschauer, M. (1997). Computer-mediated collaborative leaming: Theory and practice.
The Modem Language Joumal, 81, 470-481.
Warshauer, S. C. (1995). Rethinking teacher authority to counteract homophobic
prejudice in the networked classroom: A model of teacher response and overview
of classroom methods. Computers and Composition, 12. 97-111.
Watkins-Goffrnan, L., & Berkowitz, D. (1991). Putting grammar in its place in the ESL
curriculum. Research and Teaching in Developmental Education. 8, 21-26.
Wenden, A., & Rubin, J. (Eds.). (1987). Leamer strategies in language leaming.
Englewood Cliffs, NJ: Prentince-Hall Intemational.
Whalen, K., & Menard, N. (1995). LI and L2 writers' strategic and linguistic
knowledge: A model of multiple-level discourse processing. Language Leaming.
45,381-418.
White, A. S., & Caminero, R. (1995). Using process writing as a leaming tool in the
foreign language class. The Canadian Modem Language Review, 51, 323-329.
Widdowson, H. (1990). Aspects of language teaching. Oxford: Oxford University
Press.
244
Wilhams, N. (1992). New Technology. New Writing. New Problems? In P.O. Holt, &
N. Williams (Eds.), Computers and writing: State of the art (pp. 1-19). Oxford:
Intellect Books.
Winer, L. (1992). "Spinach to chocolate": Changing awareness and attitudes in ESL
writing teachers. TESOL Ouarteriy. 26. 57-80.
Winitz, H. (Ed.). Annals of the New York Academy of Sciences: Vol. 379. Native
language and foreign language acquisition. New York: The New York Academy
of Sciences.
Wu, H-L. E. (1991). Employing collaborative leaming strategies to assist Chinese
students in leaming written English. Dissertation Abstracts Intemational. 29,
[Online], 0365.
Yahya, N. (1994). A comparative study of multilingual writers' composing processes
(Malaysian Chinese). Dissertation Abstracts Intemational, 55, [Online], 0553.
Yin, J-H. (1995). Effects of revision strategy instmction on ESL college students' ability
to improve the quality of their writing in English as a second language.
Dissertation Abstracts Intemational, 56, [Online], 2552.
Yin, R. (1989). Case studv research: design and methods. (Rev. ed.). Newbury Park,
CA: Sage Publications.
Young, M. W. (1994). Stimulating writing in ESL/bilingual classrooms. Joumal of
Educational Issues of Language Minority Students, 13, 163-174.
Zamel, V. (1982). Writing: The process of discovering meaning. TESOL Ouarteriy,
\6, 195-209.
Zamel, V. (1983). The composing processes of advanced ESL students: Six case studies.
TESOL Ouarteriy. 17. 165-187.
Zamel, V. (1985). Responding to student writing. TESOL Ouarteriy. 19. 79-101.
Zamel, V. (1987). Recent research on writing pedagogy. TESOL Ouarteriy. 21.
697-715.
Zamel, V. (1993). Questioning academic discourse. College ESL, 3(1), 28-39.
Zamel, V. (1997). Toward a transculturation model. TESOL Ouarteriy, 31, 341-352.
Zhang, S. (1995). Reexamining the affective advantage of peer feedback in the ESL
writing class. Joumal of Second Language Writing, 4, 209-222.
245
APPENDIX A
INSTITUTO JUAN ZORRILLA DE SAN MARTEN'S
STUDY PROGRAM FOR TEACHING ENGLISH
AS A FOREIGN LANGUAGE CERTIFICATION
246
Instituto Juan Zorrilla de San Martin's Study Program for Teaching English as a Foreign
Language Certification
First year of study
English Language I
Phonetics, Diction, and Laboratory Practice I
English Grammar I
History of the Enghsh Civilization I and Geography of Great Britain
General Philosophy (in Spanish)
Second year of study
Enghsh Language II
Phonetics, Diction, and Laboratory Practice II
English Grammar II
Pedagogy (in Spanish)
British Literature I
History of the English Civilization II
Third year of study
English Language III
Phonetics, Diction, and Laboratory Practice III
English Grammar II
Teaching Methodology and Class Observation
British Literature II
History of the Enghsh Language II
Educational and Adolescents' Psychology
Fourth year of study
English Language FV
British Literature III
Contemporary American Literature
Teaching Practicum
Professional Ethics and Deontology (in Spanish)
Civic Education (in Spanish)
247
APPENDIX B
TEST TOPICS FOR LITERATURE III, AT INSTITUTO
JUAN ZORRILLA DE SAN MARTIN,
FOR THE 1997 ACADEMIC YEAR
248
Test # 1
Topics for British Lherature III class
April 17, 1997
A. Some critics believe that the theme of Great Expectations seems to be pride that leads to
a downfall that eventually produces a change in character. Do you see this theme reflected
in Pip's life? Yes? No? Why? Explain fully (Write between one page and one page and a
half).
B. Imagme you are Pip at the end of the novel Great Expectations. You have gone through
a number of experiences, and people have revealed many things that you didn't know before.
Write a letter to Joe explaining how you felt during your childhood in the village and later
when you were a young aduh in London. Also apologize for your neglect of him. (Write
between one page and one page and a half).
Test # 2
Topics for British Literature III class
September 2, 1997
1) Choose one of the following poems. Provide the name of the author. Explahi in your
own words what the main message of the poem is and then support your point by referring
to the ideas in the various stanzas. Quote parts of the poem when appropriate. Write
between 20 to 30 lines (approximately one page). Remember to organize your ideas by
providing an uitroduction, development, and conclusion.
(The text of these poems were handed out to the students taking the written test. The name
of the author was not given to them since they were supposed to know who had composed
the poems)
"Oh Yet We Tmst" (by Lord Alfred Tennyson - hi Memoriam LIV)
"The Second Coming" (by William Butler Yeats)
"Sailing to Byzantium" (by Wilham Butler Yeats)
"I Envy Not in any Moods" (by Lord Alfred Tennyson - In Memoriam XXVII)
"The Hollow Men" (by T.S Ehot)
249
Test # 3
Topics for British Literature III class
October 30, 1997
Choose one of the following topics and write a short essay (between 25 and 30 lines). Make
sure your essay has a clear thesis statement in the introduction, at least two paragraphs that
develop that thesis statement and a suitable conclusion.
1. If we compare John, the Savage, and Mustapha Mond, we can say that they represent two
opposing attitudes towards life. Explain fully.
2. Brave New World can be considered an attack on Behaviorism and the advances of
technology. Explain thoroughly.
3. The majority of the population in Brave New World lead a happy hfe and don't question
the role they have to play hi society. Why is it so? Explain thoroughly.
4. According to some critics, Brave New World constitutes a didactic science-fiction novel.
Explain what this statement means and whether you agree with it or not.
5. We see that in Brave New World traditional values have been subverted and replaced by
totally new ones. These new values are more or less opposite to the moral standards set by
the Christian tradition. Explain thoroughly.
6. Brave New World presents a seemingly perfect society. Explain the limitations of this
"perfect" world by referring to some characters' behefs and actions.
250
APPENDIX C
ANALYTIC MODEL FOR THE CLASSIFICATION
OF TEACHER COMMENTARY
251
Analytic Model for Teacher Commentary
A. Comment length (Number of Words)
1 Short (1-5 words)
2 Average (6-15 words)
3 Long (15-25 words)
4 Very Long (26 or more words)
B. Comment Types
1 Ask for information/question
2 Make a request question
3 Make a request/statement
4 Make a request imperative
5 Give information/question
6 Give information/statement
7 Make positive comment/statement or exclamation
8 Make a grammar/mechanics comment/question, statement, or imperative
C. Use of Hedges
0 No hedge included
1 Hedge included
Lexical hedges
Syntactic hedges
Positive softeners
D. Text-Specific Comment
0 Generic comment
1 Text-specific comment
Note: This is an abridged version of the original classification table hi that the examples
provided in the source have been omitted here. Source: Ferris, D. (1997) "The influence of
teacher commentary on student revision," TESOL Ouarteriy. 31. 321. Ferris' original
taxonomy is also described in more detail in Ferris, D., S. Pezone, C. Tade, & S. Tinti's
article "Teacher commentary on student writing: Descriptions and implications," published
in 1997 in the Joumal of Second Language Writing, 6, 155-182.
252
APPENDIX D
TAXONOMY FOR ON-LESfE WRITING CENTER
CONSULTANTS' COMMENTARY
253
Taxonomy for On-lme Writing Center Consultants' Commentary
A. Comment Types according to Pragmatic Intent:
1. positive comment
2. indicate, explain, or point out problem or nature of problem
3. indicate, explain, or point out problem or nature of problem + suggest solution, give
advice, or provide prompts/questions
4. indicate, explain, or point out nature of problem + provide exact word, phrase, or sentence
to be used by the student
5. only or directly suggest solution, give advice, or provide prompts/questions
6. only or directly provide exact word, phrase, or sentence to be used by the student
B. Comment Types accorduig to Substance or Issue Addressed in the Comment:
1. about the ideas/meaning/content of the essay
2. about the organization/development of the essay
3. about the vocabulary/word use in the essay
4. about the grammar and style of the essay
5. about mechanics (punctuation/spelling of words) in the essay
6. N/A
254
APPENDIX E
RATING SCALES FOR TEXTUAL REVISIONS
255
1. Rating Scale for Revisions (Ferris' original taxonomy)
0 No discemible change made by student m response to this comment
1 Minimal attempt by student to address the comment, effect generally negative or
negligible.
2 Substantive change(s) made by student in response to comment, effect generally
negligible or negative.
3 Minimal attempt by student to address the comment, effect mixed
4
Substantive change(s) made by student in response to comment, effect mixed
5 Minimal attempt by student to address the comment, effect generally positive
6
Substantive change(s) made by student in response to comment, effect generally poshive
(Source: Ferris, D. (1997). The influence of teacher commentary on student revision.
TESOL Ouarteriy 31. pp. 322)
2. Types of effects produced by revision changes made by the students in their final drafts
(New taxonomy revised for this study OR Taxonomy C)
0 No discemible change made by the student in response to comment
Definition:
The student does not seem to have altered any part of his paper in response to the on-line
writing consultant comment.
1 Change made by the student in response to comment, effect negative or neghgible
Definition: The student has incorporated, replaced, deleted, and/or rearranged words,
phrases and/or sentences producing only a minimal, slight, or imperceptible effect or an
effect that is negative because in the final product, the content is less clear or relevant to the
256
reader than it was in the original, the stmcture is grammatically incorrect, and/or the
vocabulary and mechanics are poor when compared to the initial draft.
2 Change made by the student in response to comment, effect mixed
Definition: The student has incorporated, replaced, deleted, and/or rearranged words,
phrases and/or sentences, and the effect of any of these changes is mixed because in the final
product, the content is not as clear to the reader as it was hi the original, even when the
stmcture is grammatically correct or acceptable, or because in the final product the
vocabulary, grammar, style or punctuation hasn't been improved when compared to the
initial draft, even though the content is moderately clear.
3 Change made by the student in response to comment, effect positive
Definition: The student has incorporated, replaced, deleted, and/or rearranged words,
phrases and/or sentences, and the effect of any of these changes is positive because in the
final product, the content is clearer to the reader than it was hi the original and the stmcture
is grammatically correct or acceptable. In other words, the effect is positive when m the final
product the vocabulary, grammar, style or punctuation has been improved when compared
to the initial draft and the content is moderately clear.
4 Avoidance of problem
Definition: The student has apparently avoided dealing with the problem because he/she
deleted the word, phrase, and/or sentence that was causing the problem.
5 Non-apphcable
257
APPENDIX F
TABLES WITH THE INITIAL AND FINAL NUMBER OF
COMMENTS PER ON-LINE WRITE^G CENTER
RESPONSE AS COUNTED BY BOTH
THE RESEARCHER AND
AN INTEPENDENT
READER
258
Table F.l Number of Comments per On-line Writing Center Response Counted by Both
the Researcher and an Independent Reader.
First Exchange
Second
Exchange
Researcher
Reader
Researcher
Reader
Paper # 1
10
10
13
12
Paper # 2
15
16
11
13
Paper # 3
25
23
14
14
Paper #4
22
25
18
17
Paper # 5
6
6
18
17
Paper # 6
12
13
20
21
Paper # 7
16
16
9
10
Paper # 8
7
7
8
5
Paper # 9
Total Number
of
Comments
20
21
6
8
133
137
117
117
259
Table F.2 Final Number of Comments per On-line Writing Center
Response agreed upon by both the researcher and an independent reader.
First Exchange
Second Exchange
Paper # 1
10
13
Paper # 2
16
12
Paper # 3
25
14
Paper #4
23
17
Paper # 5
6
17
Paper # 6
12
21
Paper # 7
16
10
Paper # 8
7
8
Paper # 9
Total Number of
20
7
Comments
135
119
260
APPENDIX G
EXAMPLES OF INITIAL DIFFERENCES BETWEEN
THE RESEARCHER AND AN INDEPENDENT
READER'S SEPARATION OF COMMENT
UNITS AND THEIR FINAL AGREED
UPON DECISIONS
261
Example #1
"As with the other sections of you [sic] essay, you need to support this w ith evidence from
the text. Use quotes to back up your clauns." (This comment was extracted from paper #3,
first exchange).
The researcher had marked these two sentences as belongmg to the same comment; the
reader had separated the comment following the sentence division. However, the reader had
written a question mark on the margin as a way of expressing her doubts, and upon
discussion, it was agreed that the two sentences form part of the same comment because
both sentences focus on the same problem; namely, lack of evidence and the w ay to provide
this e\ddence.
Example #2
"Explain how Pip's pride helped his dreams come tme. You cannot simply tell your readers
that this is so without giving them some type of evidence to support this claim. Use quotes
from the text to show how Pip's pride helped him achieve [sic] his desires." (Tliis comment
was extracted from paper #4, first exchange.)
These three sentences were originally considered just one comment by the researcher. The
reader, however, had divided this utterance into three different comments following the
different ideas and the sentence separation. Again, upon consideration of the main ideas
dealt with in the comment, it was decided to separate the comment into two parts: (1) the
request for explanation m the first sentence, and (2) the request for evidence through the use
of quotes in the last two sentences. The final division between comments has been indicated
in the text below by means of carets <...> and numbers at the beginning of each comment.
KExplain how Pip's pride helped his dreams come tme.> 2 < You cannot simply tell your
readers that this is so without giving them some type of evidence to support this claim. Use
quotes from the text to show how Pip's pride helped him achie\e [sic] his desires.>
Example #3
"You use a topic sentence and then follow through giving examples of each. You might
want to consider putting the 4th sentence before the 3rd. Of course, this would require some
revision so that the shift/change would make sense. But \^ou create a pattem in the 2nd and
262
3rd examples that the 1st example doesn't match; that's why I suggest the shift.
Brother-in-law needs to be hyphenated, and the phrasing 'he believes everything he tells
him' is awkward because the [sic] although 'he' is the same word, 'he believes' refers to Pip
and 'he tells' refers to the convict. Giving a name or titie for one or both of the 'he"s [sic]
might make the meaning clearer." (This long comment was extracted from paper #9, first
exchange.)
The researcher had separated this long commentary into three different comments, while the
reader had separated it into four distinct units. Upon discussing the difference between the
two divisions, the researcher admitted having overlooked the presence of a positive
comment in the first sentence, so this positive comment became another comment in the
final count. The final separation between comments has been indicated in the text below by
means of carets <...> and numbers at the beginning of each comment:
l<You use a topic sentence and then follow through giving examples of each. >2< You
might want to consider putting the 4th sentence before the 3rd. Of course, this would
require some revision so that the shift/change would make sense. But you create a pattem in
the 2nd and 3rd examples that the 1st example doesn't match; that's why I suggest the shift.
>3< Brother-in-law needs to be hyphenated,> 4 < and the phrasing "he believes everything
he tells him" is awkward because the [sic] although "he" is the same word, "he behoves"
refers to Pip and "he tells" refers to the convict. Giving a name or title for one or both of the
"he"s [sic] might make the meaning clearer.>
Example #4
"What is this 'change' in Pip's personality? I've leamed a lot from this essay about Pip's
changing. I know that he was innocent even naive at the beginning. But I don't know what
happened in London to make him materialistic, and I don't know what made him change and
ask Joe and Biddy for forgiveness. The impression I get is that there are two changes
(innocent to materalistic [sic] and then materialistic to humble). This might be good
information to add in the appropriate places. Another pronoun reference problem: does the
263
first 'his' refer to Pip or Charies Dickens?" (This example was extracted from paper #9,
first exchange.)
In this example, the researcher initially identified three comments, while the extemal reader
identified only two. The reader saw all the sentences, except for the last one, as one big
comment. She considered the last sentence a separate comment. The final separation
between comments~as agreed upon by researcher and reader-followed the researcher's
origmal division and has been indicated in the text below by means of carets <...> and
numbers at the beginning of each comment:
1 <What is this "change" in Pip's personality?> 2 < I've leamed a lot from this essay about
Pip's changing. I know that he was innocent even naive at the beginning. But I don't know
what happened in London to make him materialistic, and I don't know what made him
change and ask Joe and Biddy for forgiveness. The impression I get is that there are two
changes (innocent to materalistic [sic] and then materialistic to humble). This might be good
information to add in the appropriate places. > 3 < Another pronoun reference problem:
does the first "his" refer to Pip or Charles Dickens?>
Example #5
"You need to work on developing this paragraph morefially.Instead of simply saying that
Yeats talks about certain issues in this poem you need to show what you think Yeats'
primary claim is in the poem. What is the poem's central theme? Also, as a reader, I see no
clear connection between your first two sentences in which you discuss what you think
Yeats is saying in the poem and the last sentence in the paragraph in which you describe
Byzantium. Are you trying to show that through his description on Byzantium, Yeats is
saying something cmcial about the idea of remaining?" (This example was extracted from
paper # 6, second exchange.)
The reader separated this long comment into three units, while the researcher separated it
into only two parts. The extra comment marked by the reader was the question in the
middle: "What is the poem's central theme?" This question had been considered part of one
264
comment unit by the researcher. Upon discussing the possibilities, reader and researcher
agreed that the question constituted part of the same first comment, since it prompted the
student to indicate the author's clakn, by requestmg information about the theme of the
poem. The final division of the above commentary has been marked by carets <... > and
numbers at the beginning of each comment.
1 <You need to work on developing this paragraph more fully. Instead of simply saying that
Yeats talks about certain issues in this poem you need to show what you think Yeats'
primary claun is m the poem. What is the poem's central theme? > 2 < Also, as a reader. I
see no clear connection between your first two sentences in which you discuss what you
think Yeats is saying in the poem and the last sentence in the paragraph in which you
describe Byzantium. Are you trying to show that through his description on Byzantium,
Yeats is saying something cmcial about the idea of remaining?>
Example #6
"Why is this important? What do you think Yeats is trying to show his readers by pointing
out that people neglect monuments of ancient beauty and how does this tie in with the
concepts of eternity or lasting? You need to make this connection for your readers. Don't
assume that your readers will understand the poem in the same way that you do." (This
example was extracted from paper # 6, second exchange.)
The independent reader considered this paragraph one single comment unit, while the
researcher separated it into two different comments. Nevertheless, the reader had placed a
question mark on the margin indicating her doubts as to whether the whole paragraph w as
one single comment or more than one. The researcher's option was followed upon the
researcher's and reader's discussing and agreeing that the comment deah with two distinct
issues. In other words, the first part requests fiirther information from the student about the
content of the poem, while the second part deals with organizational issues since it asks the
student to make a connection between ideas. The final division of the above commentary
has been marked by carets <... > and numbers at the beginning of each comment.
265
1 <Why is this hnportant? What do you think Yeats is trying to show his readers by pointing
out that people neglect monuments of ancient beauty> and 2 < how does this tie in with the
concepts of eternity or lasting? You need to make this connection for your readers. Don't
assume that your readers will understand the poem in the same way that you do.
266
APPENDIX H
GRIDS FOR THE CLASSIFICATION OF WRITING
CONSULTANTS' COMMENTS AND STUDENTS'
TEXTUAL REVISIONS
267
c
c
©
u
O
a
H
c
S
o
u
<s fo
IT)
VO
00 ON
r^
f^
in
\o
00
as <s fs
U^ SO
QO OS
r^
f^
in
\o
00
ri r<5
o r^
c
OX)
OX)
c
o
u
a
B
B
o
U
268
la
a
n
m
in
00
ON
<S
f*^
in
269
so
00
ON
©
r^
APPENDIX I
TAXONOMIES CREATED FOR THE
PURPOSES OF THIS STUDY
270
(These were the three taxonomies used during the independent raters' training sessions)
1. Taxonomies for the on-line Writing Center consultants' commentary (A and B)
A. Comment Types (according to pragmatic intent)
1. Positive comment
Examples:
I've enjoyed reading your letter
This is a good beginning
2. Indicates, explains, or points out problem or nature of problem
(Informs the student of what is wrong, what is missing)
(Indicates some kind of lack or failure, some negative aspect)
(Typical sentence type: statement, but questions can occur too)
(No specific suggestion is given as to how to go about the problem, no course of action
suggested)
(No examples are given by consultant)
Examples:
- SpeUing
- This is not clear; could you explain further?
- This needs clarification / illustration / fiirther elaboration (no suggestion as to how to go
about solving the problem)
- Unclear pronoun reference
3. Indicates, explains, or points out problem or nature of problem + suggests solution by
giving advice or by providing prompts/questions
(the two parts of this category can appear in the reverse order: suggests solution by giving
advice or by providing prompts/questions + indicates, explains, or points out problem or
nature of problem)
Examples:
- This needs an apostrophe s
271
- This is not clear; could you explain why the poet feels sorrowfial?
- This is confusing; could you provide an example?
- This needs clarification / illustration / further elaboration through examples (a suggestion
as to how to go about solving the problem)
- You don't need this word here.
- You need to restate your thesis to provide a more effective end or conclusion to your essay,
(this second clause suggests something negative; i.e., that the end is not effective)
-1 think you should focus on explaining why Faustus feels he cannot repent so that you're
addressing the full question
- You want to make sure you indicate when Faustus suffers most. Example: when he sees
the two angels? When he faces the idea of an eternity in hell? etc.
- Unclear pronoun reference. What does "him" refer to?
4. Indicates, explains, or points out nature of problem + provides exact word, phrase, or
sentence to be used by the student
The word "portrait" is a noun~h's a person's picture (either photograph or paintmg).
Instead, the word needed here is "portrayed," meaning "to describe in words."
5. Only or directly suggests solution by giving advice or by providing prompts/questions
(Only IMPLICIT indication of problem)
(The sentence can be in the imperative mood, or the sentence could be phrased in the form
of a request either in the affirmative or hiterrogative forms)
(Prompts or questions often requesting hiformation or clarification)
(Suggests or shows course of action)
(May provide examples)
Examples:
- Insert a comma
- Insert an apostrophe s
- Spell "two" "too"
- Avoid this type of opening sentence (no explicit indication of problem)
- You should check the stmcture of this sentence
272
- What's your opinion?
Do you agree that Satan and humanity share common
traits/weaknesses such as pride, wrath, and envy? Are you convinced? If so, what in
Milton's poem convinced you? If not, why hasn't Milton convinced you?
- Why is the protagonist upset? What did the other characters do?
- What does "him" refer to?
- I'd like to know more about the author's ideas, (suggests that the student has not included
all the mformation and prompts hhn/her to provide some specific information)
- I am interested in learning why the poet feels happy (suggests that the student has not
included all the information and prompts him/her to provide some specific information)
6. Only or directly provides exact word, phrase, or sentence to be used by the student
Examples:
- Student's text: ...she was surprised [do you mean "shocked?"] to see the convict...
- Student's text: ...to [Maybe you mean "too" here] heavy to carry....
- You probably want to say "his obsession didn't let him think properly"
Note: Sometimes the words supplied by consultant aren't hiserted between quotation marks
273
B. Comment Types (according to substance or issue addressed in the comment)
1. About the ideas/meaning/content/development of the essay
2. About the organization of the essay
3. About the vocabulary/word use in the essay
4. About the grammar and style of the essay
Run-ons, fused sentences, comma splices
Examples:
Student's text: - We can infer/conclude [avoid this type of opening sentences]
- As we can see, [avoid these opening sentences]
- They were thinking [thought]
- He understand [s] (even if it is just one letter, this letter constitutes
an inflection)
Consultant:
- "Somebody" is not the correct word here, try "nobody" instead.
5. About mechanics (punctuation/spelling of words) in the essay
Question: What type of comment is "you need an apostrophe between Smith and the 's'?"
Answer: Grammar
6. Non-applicable
274
2. Taxonomy for students' revisions (C)
C. Types of effects (New taxonomy revised for this study)
0. No discernible change made by the student in response to comment
The student does not seem to have ahered any part of his paper in response to the on-line
writing consuhant's comment. In other words, the students' text is identical in both drafts.
1. Change made by the student in response to comment, effect negative or negligible
Definition:
The student has incorporated, replaced, deleted, and/or rearranged words, phrases and/or
sentences producing only a minimal, slight, or imperceptible effect.
OR
The student has incorporated, replaced, deleted, and/or rearranged words, phrases and/or
sentences, producing an effect that is negative because in the final product,
- the content is less clear or relevant to the reader than it was in the original (after the
consultant pointed out this problem);
or
- the stmcture is grammatically incorrect or the vocabulary and mechanics are poor when
compared to the initial draft (after the consultant pointed out these problems).
2. Change made by the student in response to comment, effect mixed
Definition:
The student has incorporated, replaced, deleted, and/or rearranged words, phrases and/or
sentences, and the effect of any of these changes is mixed because in the final product,
- even if the content is clearer to the reader than it was in the origmal (after the consultant
pointed out these problems), the stmcture is grammatically mcorrect or unacceptable; or
275
- even if the vocabulary, grammar, style, mechanics, etc. have unproved when compared to
the initial draft (after the consuhant pointed out these problems), the content is basically
unclear.
In other words:
- the student made the content clearer, as the consultant suggested, but in so doing the
grammar got crooked
- the student took care of the grammar problem pomted out by consultant, but in so doing
the meaning became unclear.
3. Change made by the student in response to comment, effect positive
Definition:
The student has incorporated, replaced, deleted, and/or rearranged words, phrases and/or
sentences, and the effect of any of these changes is positive because in the final product,
- the content is clearer to the reader than it was in the original (upon consultant's indication)
and the stmcture is still grammatically correct or acceptable,
or
- the vocabulary, grammar, style or punctuation has been improved (upon consultant's
indication) when compared to the initial draft, and the content is still moderately clear.
ALSO classify the revision effect as POSITIVE when
- the student took care of the grammar problem pointed out by consultant, but the meaning
remains confusing or unclear;
- the student made the content clearer, as the consuhant suggested, but the sentences
stmctures remain awkward or there are still misspelled words;
- the student takes care of a problem or modifies something, but not following the exact
suggestion of the consultant.
4. Avoidance of problem
Definition:
276
The student has apparently avoided dealmg with the problem because he/she deleted the
word, phrase, and/or sentence that was causing the problem.
5. Non-applicable
Note: (1) If the consuhant says "check with your teacher," on scale 3, say N/A
Note: (2) In many instances the consuhants asked the students to quote from the literary text,
but they didn't follow through with the suggestion, especially in the case of Great
Expectations, but not as often with the poems. If they expanded on the idea, though, and the
sentence stmcture was O.K., classify the change as POSITIVE.
GENERAL NOTE:
Even if the consultant's end comments or summary at the end appears to recapitulate issues
that were pointed out throughout the body of the essay, consider them as new comments (as
they have been set off).
Example: "I've pointed out some style problems throughout your paper. Make sure you fix
them."
How to classify this comment?
On scale A:
as #4 (indicates, explains, or points out problem or nature of problem + suggests solution by
giving advice or by providing prompts/questions)
On scale B (depending on the content of the majority of those "style" comments referred to):
as #4? (grammar) or maybe #5 ? (mechanics)
On scale C:
as "0 or no change" when the student has not taken care of any of these "style" problems;
as "negligible or negative" when the student has taken care of fewer than half of these
"style" problems;
277
as "mixed effect" when the student has taken care of about half of these "style" problems;
as "positive effect" when the student has taken care of most of these "style" problems;
MISCELLANEOUS:
- For POSITIVE COMMENTS, indicate type of content whenever possible.
- Could you please write number of student paper on yellow sheets? ft would also be nice if
you wrote something like "Pip" and "Poems" on the correspondmg yellow sheets.
- British Spelling
- Corrections that you may see in black mk: it means that's the way the student's first draft or
final versions were.
- Write down all comments you want
- Sometunes in the drafts, you'll see the student addressing the consultant
-1 would suggest you use an abridged version of the comment rather than the number, if it is
not too much trouble
You'll sometimes find funny characters in the writing center consultant's comments:
- You may find an intmsive R or S in some responses. These R's and S's stand for "
(quotation marks), e.g.: RmannersS =====> "manners."
- You may also find an intmsive U or T in some responses. These U's and T's stand for '
(an apostrophe), e.g.: JohnUs =====> John's or JohnTs
278
> John's.
APPENDIX J
EXCERPTS FROM FIELD NOTES
279
March 18, 1997.
Today classes began at Juan Zorrilla de San Martin school, m Cordoba, Argentma. I am
teachmg British Literature II and British Literature III. These U\ o subjects deal with n\'o
periods in the literary history of Great Britam. the periods from Renaissance to Romanticism
(British. Lit. II) and from Victorian Literature to contemporary literature (British. Lh HI). In
these two classes, the students are supposed to read plays, novels, and poems representati\e
of the hterary history extendmg from the 15th to the 20th century. Students are tested and
graded on the reading of the hterary works, especially by means of written assignments.
Today, on our first day of classes, without givmg them too much detail. I announced to the
students that I would be coUectmg data for my research project. I told them that I am
interested in studying how EFL/ESL students approach writing and how they leam how to
write in Enghsh. They paid attention to all I was saymg, but they didn't ask me any
questions about my project, even though I encouraged them to do so. They may have
refrained from asking questions because they may have perceived my unwilhngness to
reveal/uncover the main purpose of my research.
March 29, 1997.
We don't have access to Intemet yet, so we don't ha\'e access to e-mail services either.
Around March 1,1 submitted a formal letter requesting the school authorities that they
obtain an Intemet connection, explairdng why I needed/wanted it. There were some doubts
about how expensive it would be for the school to have this service, so they asked me to
submit a longer article explaining how I would use the ser\'ice and why. Therefore, I
submitted a summary of my dissertation proposal in Spanish so that the school board of
directors and academic authorities would agree to pay a fee to an Intemet provider so that
we can have access to e-mail at school. I had talked about the project with the director of
the school in October 1996, but I submitted a formal written proposal to her on March 13.
1997. I haven't received any formal reply to my petition to date. How e\ er. the dfrector has
told me that it is quite probable that my petition will be granted, e\'en though her
announcement is unofficial. Both the delay in the response and the unofficial announcement
of the approval of my proposal are things that we can normally expect in Argentina.
280
April 8, 1997.
Today I explamed to my students some more about my project. I told them that I will be
studymg how they write in English. I didn't tell them, though, what I will be looking for
when I read their compositions, which is what type of changes they make on their
compositions upon receivhig feedback from on-line writing center tutors. I also explained
the procedures I will follow to gather the data, and I asked those who wanted to participate
as subjects m my research to sign a permission form that would allow me to use their texts
for the purposes of writing my dissertation and presenting papers at conferences. Most if
not all of those students present gave me that permission. Some of them were absent today,
so I didn't have the chance to ask them for their permission. I hope to do so in the next week
or so. These are the procedures that I mtend to follow when collectmg the data and that I
explained to them:
The students taking British Literature II (approximately 25 students) will write a number of
short compositions (or rather paragraphs) during the academic year, about a character or a
character's confhct in one of the literary works they are supposed to read. They will write
those compositions in longhand. I will take their drafts home and bring them back the
following class so that the students can type them up and send them through e-mail to an
on-line writing center in the US. There is one mam reason for the students' writing their
compositions in longhand first. This written assignment will be evaluated; that is the
students will receive a grade for it, so most of the students who are not familiar with
computers and/or typing will probably feel uneasy or anxious about having to practice a new
skill such as computer operation and typing in an exam situation. So, in order to ease thefr
anxiety and dispel any fears stemming from computer phobia, I decided (and they agreed)
that I will have them write in longhand, as they are used to. They know that I will not be
grading this first composition. This first draft will be sent to the Texas Tech University Online Writmg Center, and after the students receive the feedback from the On-lme Writing
Center consultants, they will have the chance to revise their work and then tum it in for a
grade.
I told exactly the same thing to the students taking British Literature III (9 students).
The only difference is that they will be writing longer composhions. They will write a short
essay consistmg of two or three paragraphs. I also told both groups that we will go to the
281
computer room once before they actually go and type their paragraphs or essays so that they
leam the basics about how to operate a word processor. We will do so in the following
week or so.
April 10, 1997.
I announced to my students that we will go to the computer room the following class, on
Tuesday, April 15th to practice writmg something on a computer. They will have the
chance to use a word processor and save whatever they type. The students asked me some
more about the project. They didn't ask me what I was trymg to study or investigate, but
they were concemed about who would be reading their compositions. They showed some
uneasiness or discomfort at the idea that people in the USA were going to read their
compositions. They asked me whether other students like them or teachers would be
reading their writing. Some of them expressed their concem by saying something like:
"What are they going to say there about us?" I told them not to worry because these tutors
or consultants read hundreds of compositions and they are used to reading all sorts of
mistakes. I also explained that they would not have to see those tutors face-to-face, so there
was no reason to worry. They didn't show reluctance to participate hi the project, but I
could sense some kind of uneasiness about the whole thing, about having their work read
and maybe criticized by experts. They were also very concemed about their having to type
up their compositions on a computer the day of the test. Most of these students don't know
how to type, much less how to operate a computer, so they were quite troubled by the whole
idea. I reassured them by telling them that they would go to the computer lab one day
before the test so that they would try typing something (not their compositions) on a
computer keyboard. They still felt it would be too stressful to type on a computer something
that was going to be graded when they knew ahnost nothing about computers. Therefore, I
decided to have them write their compositions m longhand the day of the test. They will
tum in their written product that same day, but I will bring those compositions back on the
following class so that they can type up what they have written m longhand. In other words,
I will collect their compositions so that they can't work on them at home. I'll bring their
work intact, without corrections for them to type the following class. After their documents
are saved on a computer disk, I will send everything to the Texas Tech University On-line
282
Writing Center via e-mail. Today I also obtamed permission to use the students' w ritten
work from those students who were absent last class.
April 15th, 1997.
Both my classes went to the computer room today. The computer room has t\\ enty
networked computers. The students worked in groups of two and three around one
computer. The computers had all been tumed on before the students got in. I gave general
directions to the whole group as to the use of the keyboard (only how to use some keys) and
as to the parameters of their assignment. I asked the students in British Literature II to write
sometiiing about Dr Faustus' conflicts in the play Doctor Faustus. and I asked the students in
British Lherature III to write something about Pip, the protagonist of Great Expectations, as
a way of brainstorming and leaming at the same time how to use a word processor. I went
around the room checking their activities. I helped them save the written text. I am aware
that this practice is not enough. Some of them are not famihar with the use of a word
processor yet, and they will need more practice. So the day they type their paragraphs or
essays will not be the day of the test. I mean, they will write their first draft of the essay in
longhand. They will do so on Thursday April 17.
The students will have one hour to write their assignment. In the rest of the class period, we
will begin with a new unit. I will not grade these assignments until they submit a new
version of it, after the students have received feedback from on-line writing center tutors at
TTU.
April 17, 1997.
Today my students in both courses Lh II and Lit III took their first test. They wrote their
short essays. The prompts for the students were the following:
Topics for Literature II class:
Choose between:
A. What is the tragedy of Dr. Faustus? What leads Dr. Faustus to etemal damnation? What
is his conflict of conscience? Why is it so difficuh for him to repent? Explain ftilly. (Write
between fifteen and twenty lines).
283
B. Imagine you are Dr. Faustus. You have afready signed the pact with Lucifer, but now
you are facing an inner stmggle, a stmggle of conscience. Write a letter to your fiiends, the
scholars, explaining your timer dilemma. (Write between fifteen and twenty Imes).
Topics for Literature III class
A. Some critics beheve that the theme of Great Expectations seems to be pride that leads to
a downfall that eventually produces a change in character. Do you see this theme reflected
in Pip's Hfe? Yes? No? Why? Explam fully (Write between one page and one page and a
half).
B. Imagme you are Pip at the end of the novel Great Expectations. You have gone through
a number of experiences, and people have revealed many things that you didn't know before.
Write a letter to Joe explaining how you feh during your childhood in the village and later
when you were a young aduh in London. Also apologize for your neglect of him. (Write
between one page and one page and a half).
The students were given approxunately one hour and a half to complete their writhig task. It
was an open book test. In other words, the students were allowed to check both their class
notes and the literary works about which they were writing. Some students were very
concemed about this task. They wanted to do a good job, probably because they were
thinking in terms of grades, since by obtaining a grade higher than 7 (on a 1 to 10 grading
scale) in four of these short tests, they would not have to sit for a formal written exam at the
end of the school year. (They would pass the class by obtaining a passing grade on all of
these tests and by having a short talk about the works read, by the end of the year.) I
reminded them that they would have another chance at revising their work after receiving
the comments from the writing consultants in the US. They felt somehow relieved at the
fact that they would have another chance. Some of them felt somewhat anxious or uneasy at
the thought of having native speakers of English read what they might consider bad-quality
or ineffective writmg. They somehow feel that natives speakers would laugh at them and
their work. Only one of my students in the Lh II class was absent today.
284
April 22, 1997
Both my classes went to the computer classroom today. I had told them to arrive as early as
possible so that they could start typing their compositions right away. There are twenty
computers (and not all of them work properly!) in the computer classroom and I have 26
stiadents in my Lit II class, so we would be too many to work comfortably. As soon as they
started arriving, I placed students at a computer terminal. The room got packed after a while
and the activity was a little messy. Even though the students had practiced working on the
computers the previous week, I had to help them one by one to save their material. A
serious problem arose when I wasn't able to save some students' material onto the server. I
was able to save approximately half of the students' compositions on the server. I followed
the lab assistant's instmctions. He had explained a number of things about the network a
week earlier, but I didn't remember all he had demonstrated, so I made a number of
mistakes. In order to cope with this difficulty, I decided to go to each computer terminal and
save the students' work on one of the several blank 3 and V2" diskettes I had with me. This
was impossible, however, because only one of the computers has a 3 and 1/2 inch drive.
Consequently, I tried to save their compositions on a 5 1/4 inch diskette, but I wasn't able to
do so because my diskettes were not formatted (my mistake). I also tried, but I wasn't able to
format them in any of the computers on the network, so I tried to have them formatted in the
administration office. The monitors and administrators at school didn't allow me to format
them on their computers because they were afraid of vimses. They don't allow anybody's
diskettes on their computers!!! Thanks to a student who lives nearby, I was able to get hold
of a formatted 5 1/4 diskette and save most of the compositions that evening. Now, the
problem was that all of the students were requiring my presence to help them save their
material or help them with different characters, punctuation marks, keyboard functions, etc.,
so that I ahnost didn't have time to take care of everythmg. At one point, there were five or
six students wahing for a computer so that they would be able to type their compositions.
The situation was nerve-wracking. I intended to continue with my class after the students
typed their work, but h was impossible because I kept on working trying to save all of their
compositions so that I would be able to send them to the Texas Tech University On-line
Writing Center the following day. The students from my other class (Lit. Ill) started
arriving even when there were still students remairting from my Lit. II class. I felt
285
overwhehned. I got some of these students started and continued in the process of rescuing
as many files as I could from my Lh II students. Things were cahner in this second period.
However, I had to look for some of the students' compositions on the different computers
because, as I said before, I hadn't been able to save everything onto the server....Well, my
last class finishes at 10:25 p.m., but I remained in the building until 12:00 a.m. and I had not
finished....So the following day (April 23), I went back to school. The monitors allowed me
to use the computer room, and I finished saving all of my students' compositions on a
diskette that I later took to the place where I have a modem and an Intemet connection.
To avoid any word-processmg incompatibility problem that would not allow me to
open the documents later, I saved every composition both as a text file and as the default
word processor file the students were using.
286
APPENDIX K
STUDENT CONSENT FORM
287
Consent from the students at the teacher-traming college "Juan Zorrilla de San Martin"
(Cordoba, Argentina) to participate in a research study about writing, and the researcher's
promise to use the subjects' information in Ihnhed ways.
I, Lihana B. Anglada, would like to obtain your signed consent to participate in this research
project about EFL students' writing abilities. Before you sign this document, however, you
need to know that Liliana B. Anglada, the researcher, guarantees that
- your identity will be kept confidential;
- your writmg activities and written products will be used solely for the purposes of giving
you grades and discussing (or reporting) research findings;
- the mformation gathered from this research project will eventually be used to give
pedagogical advice to writing instmctors, to suggest new routes of research, and/or to
present papers at conferences.
If you agree to participate, please sign and date this document below:
Researcher's signature
(Lihana B. Anglada)
Research Participant's signature
First and Last Name:
288
APPENDIX L
FIRST SURVEY
289
First Survey
Lihana B. Anglada
Writing Research - Survey
The purpose of this survey is to identify advanced EFL students' attitudes and activities
when they rewrite paragraphs, letters, compositions, or essays.
I would appreciate your answering these questions about your knowledge of the English
language and your activhies when you write in English. Answering these questions will
take you approximately thuty to forty-five minutes.
A. Your knowledge of English
In this section of the survey, you will be answering some questions about your knowledge of
the English language. Please answer the following questions by circlmg the choice that, to
your knowledge, best reflects your background and experience in the use of the English
language.
1) Did you study English prior to becoming a student at "Instituto Juan Zorrilla de San
Martin?"
Yes
No
2) If the answer to question 1) was "yes," did you study English (circle all the items that
apply)
abroad?
at an elementary school?
at a high school?
at a private language center?
with a private tutor?
3) If you had to interact with a native speaker of English, how well do you think he/she
would understand you? In other words, how do you rate yourself as a speaker of English?
excellent
average
average to low
poor
4) If you had to interact with a native speaker of English, how well do you think you would
understand him/her? In other words, how do you rate yourself as a listener of English?
290
excellent
average
average to low
poor
5) If you had to read a letter that a native speaker of English has written, how well do you
think you would understand him/her? In other words, how do you rate yourself as a reader
of English?
excellent
average
average to low
poor
6) If you had to write a letter to a native speaker of English, how well do you think he/she
would understand your writing? In other words, how do you rate yourself as a writer of
English?
excellent
average
average to low
poor
B) Rereading your first draft
In this section of the survey, you will be answering questions about your attitudes and
activities when rereading a first draft before you tum in your written work to be graded.
Please circle the choice that, to your knowledge, best reflects your actions when rereading
your first draft.
1) Do you ever reread your written assignments (paragraph, letter, composition, or essay) in
order to check the content before your tum them in?
always
sometimes
seldom
never
2) Do you ever reread your written assignments (paragraph, letter, composhion, or essay) in
order to check the organization of ideas before your tum them in?
always
sometimes
seldom
never
3) Do you ever reread your written assignments (paragraph, letter, composhion, or essay) in
order to check the grammar before your tum them in?
always
sometimes
seldom
never
4) Do you ever reread your written assignments (paragraph, letter, composition, or essay) in
order to check the spelling and punctuation before your tum them in?
always
sometimes
seldom
291
never
5) Do you think h is worthwhile to make changes after you have written the first draft of a
paragraph, letter, composition, or essay?
Yes
No
6) Has a classmate ever read your written assignments before you tumed them in because
you asked him/her to do so as a favor to you?
always
sometimes
seldom
never
7) Has a classmate ever read your written assignments before you tumed them in because
he/she was required to do so as part of a class assignment?
always
sometimes
seldom
never
8) If answer to question # 7 was "always," "sometunes," or "seldom," did your classmate
suggest any changes?
Yes
No
9) If the answer to question # 8 was "yes," did you incorporate any the changes your
classmate suggested?
Yes
No
10) Have your teachers of English (non-native speakers) ever read your written assignments
before you tumed them in?
always
sometimes
seldom
never
11) If the answer to question #10 was "always," "sometimes," or "seldom," did your teachers
suggest any changes?
Yes
No
12) If the answer to question #11 was "yes," did you incorporate any of the changes your
teachers suggested?
292
Yes
No
13) Has an English teacher (a native speaker) ever read your written assignments before you
tumed them in?
always
sometimes
seldom
never
14) If the answer to question #13 was "always," "sometunes," or "seldom," did that teacher
suggest any changes?
Yes
No
15) If the answer to question #14 was "yes," did you incorporate any of the changes that the
teacher suggested?
Yes
No
16) Have you ever heard or read a comment from a classmate, thought of mcorporating it
into your written assignment, but didn't?
always
sometimes
seldom
never
17) Have you ever heard or read a comment from your teachers of English (non-native
speakers), thought of mcorporating it into your written assignment, but didn't?
always
sometimes
seldom
never
18) Have you ever heard or read a comment from an English teacher (native speaker),
thought of incorporating it into your written assignment, but didn't?
always
sometimes
seldom
never
19) How useful do you think it is to have a classmate read your written assignments and
make suggestions for improvement before you submit them to be graded?
extremely useful
very useful
somewhat useful
not useful at all
20) How useful do you think it is to have your teachers of English (non-native speakers)
read your written assignments and make suggestions for improvement before you submit
them to be graded?
293
extremely useful
very usefiil
somewhat usefiil
not useftil at all
21) How useful do you think it is to have an English teacher (native speaker) read your
written assignments and make suggestions for hnprovement before you submit them to be
graded?
extremely usefiil
very useful
somewhat usefiil
not useful at aU
C. Making changes in your first draft
In this section of the survey, you will be answering questions about your attitudes and
activities when makmg changes m a first draft before you tum in your written work to be
graded. Please circle the choice that, to your knowledge, best reflects your actions.
1) When you have the opportunity of writing second drafts, do you ever change the content
of your first draft?
always
frequently
occasionally
never
2) When you have the opportunity of writhig second drafts, do you ever change the original
organization of ideas of your first draft?
always
frequently
occasionally
never
3) When you have the opportunity of writing second drafts, do you ever change the
grammar stmctures you originally used in your first draft?
always
frequently
occasionally
never
4) When you have the opportunity of writing second drafts, do you ever change the
mechanics (spelling and punctuation) of your first draft?
always
frequently
occasionally
never
5) When you have the opportunity of writmg second drafts, do you concentrate on
improving the content rather than the overall organization of ideas?
always
frequently
occasionally
never
6) When you have the opportunity of writmg second drafts, do you concentrate on
improving the overall organization of ideas rather the content?
294
always
frequently
occasionally
never
7) When you have the opportunity of writhig second drafts, do you concentrate on
improving your grammar rather than mechanics (spelling and punctuation)?
always
frequently
occasionally
never
8) When you have the opportunity of writing second drafts, do you concentrate on
improving mechanics (spelling and punctuation) rather than the grammar?
always
frequently
occasionally
never
9) When you have the opportunity of writhig second drafts, do you concentrate on
improving the content rather than the grammar and mechanics (spelling and punctuation)?
always
frequently
occasionally
never
10) When you have the opportunity of writing second drafts, do you concentrate on
improving the overall organization of ideas rather than the grammar and mechanics (spelling
and punctuation)?
always
frequently
occasionally
never
11) When you have the opportunity of writing second drafts, do you concentrate on
improving the grammar and mechanics (spelling and punctuation) rather than the content?
always
frequently
occasionally
never
12) When you have the opportunity of writing second drafts, do you concentrate on
improving the grammar and mechanics (spelling and punctuation) rather than the overall
organization of ideas?
always
frequently
occasionally
never
Name (optional):
Thank you very much for your time and cooperation hi answering this questionnaire.
295
APPENDIX M
SECOND SURVEY
296
Second Survey
Liliana B. Anglada
Writing Research - Survey
The purpose of this survey is to identify attitudes and activities advanced EFL students
engage in when they rewrite paragraphs, letters, compositions, or essays, upon receiving
feedback from an On-line Writing Center consultant.
Answering these questions will take you approximately twenty to t\\ enty-five minutes.
A. In this section of the survey, you will be answering some questions about your
opinions and reactions to the project you have participated in during this school
year. Please answer the following questions by circling the choice that, to your
knowledge, best reflects your actions.
1) How many times were your compositions sent to an On-line Writing Center?
once
twice
three times
more than three times
2) Did you incorporate any of the suggestions coming from the On-line Writing Center?
yes
no
3) If the answer to question # 2 was "yes," approximately what percentage of suggestions
did you incorporate?
0-25%
26-50%
51-75%
76-100%
4) If the answer to question # 2 was "yes," why did you incorporate those suggestions?
Check all that apply by placing a tick (a mark) by the blank in front of each sentence.
Then rank your reasons using 1 by the reason you marked that you consider most
important, 2 by the reason you consider second in importance, and so on.
Because you thought you would get a better grade if you incorporated them.
Because the suggestions came from a native speaker of English.
Because the suggestions seemed reasonable.
Because the suggestions seemed simple and/or easy to do.
Because the suggestions seemed not to require much time to do.
None of the above.
5) How useful do you think it was to have an English teacher (native speaker) read your
written assignments and make suggestions for improvement before you submitted them
to be graded?
297
not useful at all
somewhat useful
useful
extremely useful
6) Which of the following activities do you consider useful and/or valuable for improving
a first draft? Indicate all that apply by placing a tick (a mark) by the blank in front of
each sentence. Then rank the order from least useful to most useful by using numbers
(for example, 1 for "not useful at all," 2 for "somewhat useful," 3 for "usefiil," 4 for
"extremely useful")
To have a classmate read and comment on your first draft.
To have an English teacher (a non-native speaker of English) read and
comment on your first draft.
To have an English teacher (a native speaker of English) read and comment
on your first draft.
To have nobody else but yourself to read and comment on your first draft.
7) How would you describe having to use computers to type up your draft before being
able to send it and receive feedback? Indicate your opinion on both scales below:
Scale One
not helpful at all
somewhat helpful
helpful
extremely helpful
somewhat annoying
annoying
extremely annoying
Scale Two
not annoying at all
8) Considering that you did not have an Intemet connection, how would you describe
having to resort to your teacher (as an intermediary) to send your composition and to
receive feedback?
not problematic at all
somewhat problematic
problematic
extremely problematic
9) Indicate your feelings about waiting for a response from the On-line Writing Center on
both scales below:
Scale One
not patient at all
somewhat patient
patient
extremely patient
somewhat anxious
anxious
extremely anxious
Scale Two
not anxious at all
298
B. In this section of the survey, you will be answering some specific questions about
the feedback and suggestions you received from the On-line Writing Center
consultants. Please answer the following questions by circling the choice that, to
your knowledge, best reflects your thoughts and actions.
1) To what extent did you understand the comments and feedback from the On-line
Writing Center consuhants?
0-25%
26-50%
51-75%
76-100%
2) Did you need the help of a classmate to understand some of the comments you
received?
yes
no
3) Did you need the help of a teacher to understand some of the comments you received?
yes
no
4) Overall, how useful do you consider the feedback you received?
not useful at all
somewhat useful
useful
extremely useful
5) Which type of comments or suggestions from the On-line Writing Center consultants
did you ignore? Check all that apply.
Content
Organization of ideas
Grammatical stmctures
Mechanics (spelling and punctuation)
None
6) Which type of changes did you incorporate most frequently? Check all that apply by
placing a tick (a mark) by the blank in front of each sentence. Then rank the changes you
incorporated by using 1 by the type of change that you made most frequently, 2 by the
type of change that you made second most often, and so on.
Content
Organization of ideas
Grammar stmctures
Mechanics (spelling and punctuation)
7) Were there times when you considered making changes suggested by the On-line
Writing consultants, but you didn't?
299
i^sver
seldom
sometimes
many times
C. In this section of the survey, you will be answering a few questions about your
overall reaction to the idea of receiving feedback and suggestions from On-line
Writing Center consultants. Please answer the following questions by circling the
choice that, to your knowledge, best reflects your thoughts and beliefs.
1) Do you think your writing has improved because you have received feedback from online writing center consultants?
yes
no
2) Do you think your writing would improve if you could send your compositions to and
receive feedback from on-line writing center consultants on a regular basis?
yes
no
3) Would you recommend that other students send their compositions to an on-line
writing center for feedback?
yes
no
4) If your school gave you the chance of having your written assignments read by an online writing center consultant, would you take the time to type your composition, send it,
and wait for a response?
yes
no
Name (optional):
Thank you very much for your time and cooperation in answering this questionnaire.
300
APPENDIX N
INTERVIEW QUESTIONS
301
Interview Questions
1. Have you ever had a person read your composition before you tumed it in for a grade?
2. What do you think of this way of getting feedback from instmctors in the USA?
3. What did you expect before actually getting the comments?
4. Did your opinion change after receiving the comments?
5. How useful were the comments?
6. What is more important for you, help in grammar or help in the organization and
content?
7. How did the instmctors in the USA help you more?
8. At which level do you think this type of help would be more useful? at a beginner's
level, at an intermediate level, or at an advanced level?
9. Is this kind of exchange useful for a literature class? Would it be more useful for a
language class?
10. Is it worth rewriting your composition?
11. What is the best thing about this exchange of information, this feedback?
12. What is the worst part of it?
13. If you had the opportunity to send your composition again, out of your own decision,
would you do it?
14. Were computers a hindrance during this project?
302
APPENDIX O
CLASSIFICATION OF COMMENTS ACCORDING TO
PRAGMATIC INTENT AND STUDENTS'
TEXTUAL REVISIONS FOR
THE FIRST EXCHANGE
303
2
•*->
<
g
M-(
<u
O
(U
>
u
•(-»
bX)
03
<4-l
o
ID
o
_>
•
o
o
1-H
(/I
<
<
<
g
g
g
C".
f,
~—'
•4—1
03
4=
o
Z
-t—)
^-l
m
^ '
-i«—1
(U
CJ
(U
(U
CJ
>
_>
-1—'
•4—'
>
'.^
c/:i
o
0^
OH
-t—>
D
bJQ
(U
O
<u
o"
Xi
Z
CO
X
m
o
•t—>
-(—<
o"
fee
g
m
fee
<
fee
Rat
c
<
fee
PQ
55
CJ
c/j
O
O-
o
OH
O
CU
m
r<-)
m
u
•4—>
c3
>>.
o
B
<
g
g
r-i
•*->
<
g
o;
m
<u
•4—>
C3
u-i
<u
<u
_>
.x:
o
•t—'
;2^
o
M
o
t+H
<u
o"
i>
<
<
<
g
g
g
u-i
ij
(U
c3
42
(U
CJ
>
o
:z;
•
>
I—«
•4—>
CO
crt
O
o
OH
OH
-4-<
^
-4—"
fee
H
^3
<
fee
^
fee
!/)
03
fee
.1—1
fee
axono
IS
-4—1
tH-(
OJ
(U
_>
*
J
'M
OH
o
OH
in
U~i
<U
<u
>
•4—'
'c«
O
OH
a
^
(N
u
Rate
ao
c
o
X
4-'
(U
<u
e
e
B
B
o
o
H
<u
_>
'c^
CJ)
X
e
e
o
o
s
^
3O
(U
(U
>
Vj
-*->
1/3
CO
O
C3
OH
:^
o
OH
(U
V.I
V-i
)_i
u
a
a
o
a
a
o
a>
(U
(U
(U
.>
_>
.>
-*-»
^-l
'+->
o
•
o
1—t
3c3
3C3
<+H
O
^
:^
o
OH
<
>%
a
a
o
o
CO
C/3
CO
o
OH
O
pL,
c:
^
13
-4—>
CX
^
"Td
•*->
a
o
CX
CX
CX
a
+
o
o
—
a
PLH
o
Ul
CX
t4-l
o
03
03
Z
:^
o
O
^
^
robl. +
xact
o
p^
..—1
• 1 - ^
C/3
O
OH
o
i_i
B
(U
>
• 1-H
a
CO
<4—1
O
OH
4-1
w
1-^
<S
m
Tt
(U
(U
+
, ;
X
O
_>
a
<u
a
a
o
o
(U
>
>
•4-<
c«
O
O)
C/3
O
O
eu
PH
OH
00
ON
O
•
1—«
r-
^"
a o
H ;2i
304
i>n
ir>
13
-4->
• 1.H
'-4->
^
O
^
PQ
o
o
'^
'rt
•4—>
-4—<
a
a
o
u
CX
a
o
1-
OH
OH
«S
m
^"
1-H
cd
^
IT)
c(U
o
a
a
:^
-4->
:z
;^
'-^ --5
:3^
•4—>
a
Ui
o
^
(U
ofp
^
^
<u
Nat.
<u
>
• 1.H
(/3
C/3
O
(U
'-*->
. I-H
^3
s
m
••->
4->
+
>
o
o
o
^
'^
(U
com
s
i^;.
-t->
C
<u
m
ofp:
««
p!i
^
com
H
-^
com
C3
©
UIO
B-B
CX
•4—>
<u
CX
o
•4—'
(U
o
ro
-4-»
ter A
(/}
•4—>
a
o
B
.«—>
•i->
+
o
o
I—1
xact
O
1-H
-*->
• 1 - ^
c<u
B
(U
3
1—1
•4—"
com
o
o -^
o
o
PLH
>
•
-(->
m
com
fl
•<—>
o
o
(U
op dJ
B
w
3
(N
(N
obi
P9
^
^*
m
-4_>
cd
oi
^^
1-H
^"•^
<4-4
^^
(^
^
—^
•4—<
^_^
^_^
ro
r^
r^i
^ • • - ^
-4—>
CJ
O
«
<u
•
?^
t ^ - l
&p
'3)
c
^-^
^—^
•4-'
CJ
^
^ l - l
(U
(U
(U
(U
(U
(U
>
_>
_>
VJ
•4-»
-4—"
C/3
c«
^^
ro
•4—>
o
^
^OJ
•4—'
o
^
CH
aj
_>
-4—>
^^^
m
•4—'
CJ
c^
CH
OJ
(U
(U
(U
>
_>
-4—<
•4—'
O
O
O
O
O
o
OH
OH
OH
OH
OH
OH
^
^ • ^
,^-s
c<-)
^-s
CO
^*-^
•4->
^-^
m
^~~^
m
v^.^
^-^
^—^
r^
o
^_^
^^
^ ^
fN
m
r'^
•4—>
o
^
CH
<u
T3
(U
X
S
, ^ j
•4—>
CJ
,'~>
'- 1
1
CJ
<u
>
•4—*
CJ
a
, ^
<
r^>
Z^
^
—^
•4.J
z
u^
<D
<u
>
<
g
o
5^
<u
<N
^H..^
•4-'
o
^
t+H
tl-(
(U
(U
TD
>
'•J
X
-4.J
• 1"H
•4—*
^^
CO
O
o
OH
OH
OH
^-^
m
^—^ <
.-—
^_^ g
.
^^
«—(
^_^
2
o
>
•4—>
cd
e>a
B
o
<
©
u
H
(U
r^
•4—>
•4—*
Cd
oi
^**-^
o
-4-»
•N.-^
-4-'
CJ
'-^m.y
•4-'
<u
,>
-4—>
• 1.H
C/5
o
OH
<u
<u
_>
-4—>
aj
(U
(U
OJ
(U
(U
,>
.>
.>
'-4->
•4-'
'-4->
O
CO
O
c«
M
OH
OH
OH
OH
C/D
o
o
-4—>
(U
(U
u
tofl
C
Xed
I
O
t+1
OJ
>
_>
CO
CO
OH
OH
•
^^
o
-4—<
CJ
OH
(U
•4—>
ro
1—1
•4—>
O
o
CJ
(U
OJ
>
• 1-H
O
OH
OJ
-4->
Cd
0^
O
+
3
o
i-i
CX
o
•4-*
cd
:^
<
>%
m
o
+
3
^
Id
-4->
CX
a
o
o
CX
OH
<+H
CO
m
-4—>
-4->
a
o
a
o
CX
I-I
+
^^
o
I-I
CX
o
CX
+
I~-H
o
in
^
CX
a
o
I-I
OH
o
'iZ,
-4->
c3
s
o
+
2
o
CX
•4—<
^
Id
•4—>
I-I
I-I
o
o
<U
OJ
aj
>
>
>
-4-'
•4—'
-4—"
m
cs
O
a
T3
CX
o
I-I
^
_^
JO
OH
OH
o
I-I
a
+
a
;z;
cd
o
o
I-I
03
Z
s
CX
-4-'
^
r^
CX
a
CJ
O
CX
^
Id
i^)
I-I
03
03
W)
&J0
(U
<U
Z
—
•4—>
o
«4-(
o
m
I-I
CX
•4-»
IT)
OJ
I-I
^-I
03
^
' *"!
&0
:z;
'^
^.-^
si)
' *"!
bfi
^^
^^
"cj
si
M)
;J3
03
CJQ
OJ
^
g
"oj
CO
o
<
m
2:
—
•4—»
-4—>
c
a
a
OJ
o
CJ
(U
-4—>
C+H
o
o
OH
-4-'
•4—>
-4—*
CX
c:
B
o
a
a
o
t-l
CX
+
X 5'
o
1CX
•
03
B
o
CX
+
>
O
•4—>
—.1
CO
OH
•4-'
'
I-I
CX
«+H
o
-4-'
03
z
CX
o
O
o
m
cd
"A
Z
o
B
©
<
^
^
^
-4-1
•4—'
Cd
>-l
O
+
mtinued.
O
O
+
o
i-i
CX
<+H
o
VH
CX
o
I-I
CJ
03
X
m
U
TT
IT)
3c i
ot
o
1-4
a
t+H
O
CJ
a
X
w
3
-4->
in
z
—
O
O
4-
•4—'
iB
o
a
i-i
-4-'
a
a
o
o
03
X
w
-4—>
a
a
o
I-I
I-I
OH
OH
a
a
o
CJ
_>
'S5
o
OH
03
ON
©
H H ^
305
--
a
OJ
a
a
o
o
)-l
CX
CJ
+
>
•
I-I
o
CX
t+H
o
1—1
•4—'
CO
O
OH
•4->
1-H
<S
fM
<M
f*^
TT
<S
fS
\n
«s
a
o
a
+
o
I-I
CX
tH-l
o
cd
03
fS
CX
-4-'
Z
7:
O
m
-4—»
'-4->
Z
00
r<-)
-4—»
I-I
o
03
cd
r-
•sO
13
13
^4-1
Z
VO
m
Id
4->
•t-i
d
j j
3
o
o
o
03
^
+
CX
o
:z:
a
I-I
<+H
-4-1
cd
a
a
Ix
^
'O
t^
r<
*s
Cd
0^
O
-4—>
0)
CJ
c
03
^
M—1
o
03
(U
rC
<u
_>
o
J3
o
-4—>
o
Z
o
c«
O
2:
OH
m
m
o
^(4-1
0
aj
aj
>
• 1.H
•4.J
So
0
^
CH
D
a>
_>
'•>->
C/3
m
0
cii
t+H
a>
0
nge(
-4—»
m
ngei
;_
cd
Jil
D
0
>
0
-4—«
tri
2;
r-1
m
r^;
0
0
CJ
ciJ
5H
aj
aj
•
>
1—1
^
CH
aj
aj
•
4—1
• I—i
c/3
^C4-1
m
CJ
^< 4 - l
aj
<D
aj
aj
m
r^
CJ
^4 - 1
^
<u
CJ
^! 4 - l
OJ
aj
aj
CJ
/^
'•H—1
CJ
13
<U
>
_>
>
_>
4->
•4.J
-t—>
4—>
• —H
4—>
X
M
1/3
CO
CO
S
1—1
• 1.H
t/J
0
•
>
fN
1—<
0
0
0
0
0
0
OH
OH
OH
OH
OH
OH
OH
OH
OH
0
ro
m
^^^
-4->
^ • ^ ^
ro
^^^
-4—>
N^*^
•4—>
0
0
CJ
CJ
CJ
E^
U
> i
<
ro
m
I-I
^ • H - ^
^^•m>y
(U
•4—>
•4-'
o
^< 4 - l
o
^! H - I
•4—"
M
Cd
0^
OJ
(U
aj
aj
_>
.>
'-4->
[-4-'
CO
1/3
m
13
-4-»
I-I
M
aj
aj
^
X)
o
I-I
CX
tl-(
o
•4-'
u
_>
•4—>
•4—'
w
0
c«
0
z
CJ
5^
<H-I
aj
aj
>
• 1.H
•4—'
• 1-H
OH
VO
'^
m
cn
-4-1
13
13
-4->
-4->
a
I-I
I-I
0
0
•4-'
+
o
+
._;
X
O
I-I
03
W
30
I-I
CX
ti-i
a
u-i
o
0
•4-'
cd
z,
:z;
VO
m
m
VO
13
O
-4->
•4-'
0
I-I
CX
I-I
CX
+
+
fO
r^
v^^
4—>
4—'
4—>
0
0
^^^H
^^4-1
^(4-4
I-I
aj
OJ
aj
aj
aj
OJ
CX
>
>
_>
•4—"
- H
4—>
'4->
CO
c^
¥
JJ
30
<4-l
•
0
• 1—1
*
OH
VO
VO
ro
13
•4—'
I-I
0
0
4-4
•4—'
CJ
03
CJ
03
X
X
w
W
CX
I-I
a
+
, \
-4—<
0
g
aj
4-<
CO
<
^
^4-1
<u
_>
• ^H
OH
•*->
>
' 1.H
1—1
0
13
aj
aj
CO
0
I-I
<4-l
r<i
C/5
0
OH
OH
OH
in
0
VO
m
13
13
4.J
0
0
4—»
4—>
0
03
CJ
03
X
X
w
w
aj
4—>
CX
a
I-I
a
+
^
X
0
I-I
CX
^+H
<H-I
«4-l
0
0
0
0
4-4
•4-'
03
z
•4-4
cd
Z
^
r*^
m
-0
I-I
13
•4->
-4-*
0
0
0
+
I-I
OH
I-I
CX
ed
Z
x^
0
0
CX
.—J
X3
0
1-4
CX
UIO
:z;
a
v.»^
aj
OH
ro
s.^^
0
0
CX
'^
CO
0
cd
B
03
x:
CJ
-4—"
OH
03
<
>^
aj
>
0
+
^U-t
aj
r^
^^.^
UIO
CX
o
X
^
ro
UIO
Rater B
VO
03
o
^^ 4 - 1
0
UIO
OH
O
03
X
o
O
OH
I-I
aj
^ H ^
UIO
O
o
^-^^^
Avoidanc
'axono
s
^
X)
0
VH
a
C4-1
03
z
©
<
X
I-I
OJ
03
inued.
w
I-I
a
a
+
, ;
X
O
JO
CX
<-^
o
]
Uo
*
03
00
<N
OJ
V3
es
+
, ;
I-I
-4-'
d
scd
I-I
X
w
o
I-I
o
03
z
03
o
fn
CX
-4->
-4-4
OS
I-I
0
<H-I
:^
30
I-I
a
+
, ;
X>
0
^4-1
CX
:z^
<s
03
1-H
fs
f*^
f*^
CX
+
^
X)
0
I-I
13
0
13
4—"
4->
4->
0
Cd
0
03
X
X
W
w
I-I
0
a
m
4—>
CX
UIO
CJ
I-I
m
UIO
•4->
CX
UIO
Cd
CX
UIO
-4-"
UIO
H
VO
VO
1-
I-I
CX
CX
+
+
,—[
1 — >
x>
0
x>
0
I-I
m
VO
VO
13
0
'O
4—>
0
UIO
c
©
0
I-.
CX
I-I
4—>
0
03
03
X
X
w
w
I-I
CX
+
,—^
X
0
I-I
CX
<-^
0
^4-1
tH-H
ti-i
0
0
0
0
-4-4
4-4
4-»
H4—»
cd
z.
03
^:
z
z
m
Tt
in
VO
fo
r-
f^
m
00
m
•i-j
03
r<5
©
H H ^;
306
ri
t d
03
Cd
z
ON
m
0
Tf
^
TT
I-I
^*H*^
m
^-^
m
aj
-4-<
4->
03
o
CQ
0^
^.v
^ H ^
OH
CHH
aj
aj
aj
aj
'4-4
•
>
.—t
4->
• I—1
^^
^ ^
0
r->
^ • H ^
aj
C>0
c
X03
0
+->
CJ
<
g
0
aj
c
03
aj
aj
X
CJ
0
Z
^^
0
z
'4-4
• 1—1
^^
m
4—>
CJ
<
g
^^
^.^
^^
0
0
r^i
4—•
^•i->
—^
.
4—>
4—4
c
CJ
CJ
0
0
C
ed
X
aj
aj
aj
aj
u
a^
0
Z
• 1.H
^^
m
^-^
4-4
0
• 1—1
m
•^—^
aj
X
>
^^
ai
03
aj
aj
, ,
m
s^.^
0
a>
0
z
>
aj
>
'4->
4—<
4—'
4—>
'cO
CO
0
0
OH
CO
CO
CO
CO
0
0
0
0
0
0
OH
OH
OH
OH
OH
OH
OH
m
^ >
m
^
1J
CO
4—"
•t->
-*-J
^4—>^
CJ
0
0
CJ
<4-l
C4-I
«4-l
aj
aj
aj
aj
aj
CO
CO
xono
u
>^.
s
n
<
;_i
u
4—>
H
03
p<
ro
CO
^4—> -'
^4 . J^
0
^-(
aj
aj
+->
ed
Oi^
g
^
U-i
m
^4-4 >
CJ
<
z
aj
aj
aj
>
• f-<
>
• 1—1
fO
aj
4—>
&0
C3
>
0
<\i
0
u
>
z
<:
7*"^
Z
^
STH
XI
0
4->
•4->
0^
03
t4H
aj
aj
4->
u
0
<
o"
<
aj
Z
&o
r""^-
03
C4-I
a>
X
0
aj
0
>
>
4—'
4—>
4—4
CO
CO
0
0
'co
• 1.H
CO
CO
CO
0
0
0
0
OH
OH
OH
OH
OH
VO
VO
m
r<-)
13
13
4-"
4-4
I-I
0
I-I
0
4->
4->
0
03
0
03
X
w
X
M
a
0
I-I
+
,—i
Xi
0
I-I
CX
<H-(
0
4-4
^
>%
fS
a
aj
X
0
i-i
CX
^
4-»
C
aj
a
a
0
m
CX
-4->
+
D
r—4
'-4-4
I-I
303
Z
OH
03
z
B
J3
I-I
CX
0
'co
0
aj
^
0
.>
in
a
0
C4-I
ai
b
CO
Id
CX
&
0
^
4-*
aj
a>
>
:^
• 1—1
• 1.H
s^,_,^
(N
a
a
a
0
X
0
CJ
<4-l
I-I
CX
a
0
I-I
CX
+
ro
>
4—»
CO
0
0
OH
OH
OH
en
r<^
VO
r<-)
4->
4-4
13
4—<
CX
a
0
1-4
CX
+
a
0
I-I
CX
+
I-I
0
-4->
CJ
03
X
CX
a
0
I-I
CX
+
1—1
1 — ^
X
0
I-I
CX
X
0
t(_l
!4-l
<4-l
0
0
0
0
4-4
4-4
4-4
4-4
•4-i
03
03
03
Z
Z
z
z
m
m
m
VO
r-)
4—»
+-4
•4—'
13
-4-4
X
0
1-1
OH
CX
t4-l
0
aj
>
•• ^I—(
-»
0
i)
1—1
;3
CO
4—4
0
03
OH
:^
I-I
a
—-^
X
0
i-i
CX
w
^-^
X
0
I-I
CX
ti-i
03
03
z
o
©
0
<
VO
m
CO
^
13
13
4->
4-"
-4->
0
0
VO
1-1
e4
aj
H
4->
Cd
oi
4->
CJ
03
X
w
13
4->
0
cd
X
W
CX
a
0
I-I
CX
+
^ ;
X)
0
I-I
:3
CX
'•4->
c
o
U
aj
et
©
«s
"^
a
0
I-I
+
^
aj
a
a
0
0
aj
x>
0
_>
CX
CO
I-I
^4-4
«4-l
0
0
•4-4
4-*
03
O
CX
4->
0
OH
z
f^
rt
in
• ^
Tt
TT
VO
-"I-
r>~\
^
4->
4->
CX
CX
0
0
I-I
a
I-I
a
a
CX
+
^
+
^
X
0
X
0
^
CX
I-I
CX
ca j
a
a
0
CJ
r^
a
aj
X
0
I-I
^
X
0
aj
I-I
CO
3
ed
<+-l
<4-l
0
0
OH
4-4
4-4
z
03
z
t>
00
as
^
-^
Tf
307
I-I
CX
a
CX
+
, ;
X
0
I-I
CX
a
0
1-4
+
^
X
0
I-I
0
4->
03
X
w
I-I
a
0
I-I
+
, ;
X
0
^
CX
«4-(
«4-l
a
<+H
0
0
0
0
4-4
4-4
•4-4
4—•
z
z
03
z
0
1-H
in
m
r<i
in
m
ed
Z
H ;^
0
1-4
0
>
4->
• w^
0
I-I
+
<4-l
• 1-H
a
a
CX
a
aj
0
03
03
z
m
a
«4-l
03
in
Cd
z
"^
in
in
in
CO
CO
>
_>
•
1—1
4->
*
4->
1—1
CO
CO
O
O
O
O
OH
OH
OH
OH
m
CO
m
^*-^
o
"^H*^
CJ
m
^*-^
o
13
aj
X
S
•
>
1—4
4—"
CO
O
OH
CJ
OJ
D
aj
13
a>
X
s
•
>
1—1
4—»
CO
O
OH
CJ
aj
O
ange
aj
aj
<D
aj
CJ
(N
X
13
aj
o
X
z
§
o
en
r^i
CJ
^—^
o
aj
aj
aj
CJ
>
• 1-H
>
4—1
CO
CO
en
en
o
CJ
^—^
ffe
1—1
4-4
• f—1
aj
aj
aj
m
ffe
•
CJ
(N
aj
CJ
aj
aj
>
_>
;-2
C
4—>
o
O
O
OH
OH
0-
OH
^
<N
O
?N
4—'
ange
-4—'
a;
a>
>
CJ
en
"•—'
ffe
<u
>
CJ
^-^
ffe
aj
CJ
ffe
ffe
^
«
<N
ffe
o
CJ
m
^-^
ffe
Rater
^ • ^
ffe
m
^—^
ffe
ro
ffe
r^
^-^
ffe
CQ
U
>^
-4—»
0^
• 1-H
>
>
CO
CO
CO
CO
s
o
Z
X
S
O
O
O
O
OH
OH
OH
OH
m
VO
m
m
m
m
in
en
in
m
in
in
en
en
13
4->
4->
4-4
-4-4
aj
aj
4-4
aj
aj
4—>
4->
^->
CX
a
o
I-I
a
I-I
O
CX
a
o
I-I
CX
a
o
1-H
a
o
OH
cd
+
, ;
+
, ;
X
O
I-I
CX
X
q-l
«4-l
X
PQ
a
o
I-I
a
+
, ;
X
O
l-H
a
a
o
1-H
a
•f
^
X
^
'^
-4->
CX
a
o
I-I
OH
CX
a
o
I-I
CX
+
, ;
X
c
^
'cd
CX
a
o
I-I
OH
a
a
o
I-I
c
^
'S
CX
•4—'
+
, ;
X
O
^
'^
CX
CX
B
B
4—4
CX
c
o
I-I
CX
CX
o
l-H
o
l-H
, ;
PLH
o
l-l
a
OH
a
+
X
o
l-H
a
+
, ;
X
O
VH
o
o
o
a
<+H
o
o
o
o
a
o
•4-4
4-J
4-4
-t-4
4-J
4-J
4-*
4—'
03
03
ed
03
03
03
03
03
4-J
4-"
CX
CX
o
I-I
c
o
1-H
a
q-i
o
ro
o
I-I
CX
•*-j
Avoidane
O
CX
«4-l
Z
z
z
z
VO
m
en
m
VO
m
m
13
•4-4
4-J
4->
u
•4-4
Z
Rater
o
aj
X
I-I
O
-4.J
o
03
a
I-I
CX
CX
a
a
I-I
o
I-I
o
CX
CX
o
o
03
I-I
CX
z
^c
'^
•4—'
CX
a
CX
a
1-H
o
a
«4-l
en
aj
4-4
Id
4->
CX
a
CX
in
aj
in
en
VO
aj
•4-4
a
a
l-H
o
a
l-l
o
4—>
CX
CX
o
I-I
o
I-I
a
a
+
^
I-I
I-I
I-I
l-l
o
q-l
«4-l
q-i
q-H
t4-H
«4-4
o
o
o
o
-t-4
-4-4
-4-4
4-J
W
X
O
X
X
M
OH
o
o
o
a
o
4-4
•4-J
4-J
za
z
z
QC
as
o
1-H
VO
VO
CX
03
Z
VO
in
r-
in
in
03
in
o
1-H
X
O
o
1-4
OH
i-i
CX
CX
03
03
©
H H ;?;
308
m
VO
I-I
o
OH
OH
CX
03
Tt
VO
in
VO
+
^
X
o
^
CX
03
X
M
«4.H
03
z
z
<S
VO
X
o
CX
+
, ;
X
+
^
-4->
03
Z
Z
in
«4-l
z
+
, ;
X
+
, ;
CX
<4-H
)P-I
Taxono
CO
)P-I
inued.
CO
<4-l
OH
o
CX
ct
CO
CX
>
O
UIO
cd
• l-H
aj
13
aj
OH
UIO
X
CO
• *—1
CJ
O
X
aj
CO
_>
X
OH
+
, ;
d
>
aj
13
aj
O
CX
o
U
>
• 1-H
I-I
CJ
OH
I-I
-4-4
aj
aj
u
>
•4—"
O
«4-l
<
aj
aj
CJ
OH
X
O
I-I
CX
S
aj
aj
aj
aj
¥
ffe
>
a>
z
ffe
* 1.H
CJ
oble:
>
Q
ffe
• 1—1
CJ
ro
m
^^^^ ^*-^
ffe
aj
(U
ro
^—^
o
ffe
aj
aj
u
cn
"^H-^
ffe
aj
m
^*-^
o
ffe
ffe
aj
aj
+
, ;
^
>%
CJ
ffe
"^•H-^
ffe
Rate
I-I
ffe
<
erB
raxono
a
z
VO
VO
rVO
X
VO
ON
VO
%
,^^
0
aj
a>
aj
aj
aj
aj
aj
aj
_>
>
• 1-H
•4—1
4—>
>
13
OJ
X
s
<
^.^
en
^-^
en
^•*^
4-4
^«^
en
^.•-^
^ ^
en
^^^
g
aj
>
-^-»
^^
,,^^
CJ
(N
. ^ j
CJ
,CJ
e^i
^-^
, ^ H
CJ
ffe
tJ
(N
ffe
4—4
0
ffe
>
-4—»
• ^H
•>->
ffe
aj
aj
en
aj
"cj
CJ
^3
aj
OJ
13
aj
X
X
>
• l-H
i
s
Pos
:s
>
•4—"
0
•4->
(N
Pos
u
X
CJ
aj
13
aj
X
Pos
13
4->
CJ
en
ffe
aj
aj
0^
0
N—.^
Pos
aj
g
en
en
ffe
CJ
m
^—»^
-t—1
Pos
CJ
(N
^•4—>
—-^
ffe
en
^*-^
-4—>
Pos
-4-4
ffe
aj
cd
(N
^ • - 1 ^
Pos
l-l
ffe
CQ
^ .
en
^ ,
r^i
, ,
^^
^—
. ^
.
u
s
<
l-l
aj
^-^
(N
" • • " - - ^
H4.^
•4—4
M
Q
03
^
54-H
OJ
XJ
u
X
s
^-v
fN
^•4—<
*—^
0
<
z
^
U-i
^-.^
0
^ H ^
aj
g
4-4
0
^
03
aj
• ^ ^ . ^
^
t4-l
aj
aj
13
aj
0
X
Z
0
_>
s
CJ
«4-H
aj
aj
X
>
•
4—"
4-J
0
1—1
4-4
CO
#
u-l
aj
aj
>
- 1-H
4—<
r<~i
4—>
^_^
.4—'
0
CJ
^
<
z
N^.^
^—•
,4_,
4-"
0
0
,<1-*
«4-l
'''-t-H
^
t(-H
OJ
aj
aj
aj
aj
aj
4J
aj
>
_>
,>
_>
4—1
-4—»
4—1
CO
CO
CO
^4-1
• 1-H
4—1
CO
^
0
0
0
0
0
OH
OH
OH
OH
OH
0
0
OH
OH
aj
ti)
i^
M
CJ
veor
'axono
>^1
'4—'
03
to
OJ
m
4-4
4—'
CX
a
0^
0
l-H
a
+
,_^
X
0
l-H
CX
a
0
CX
+
(—H
X
0
l-l
a
r*^
-t-4
-t-4
CX
aj
a
a
0
CJ
•4-4
'co
0
<+H
«4-H
0
0
4-4
•4-1
03
z
03
z
en
en
^
-4-»
4-4
•t->
OH
a
0
l-H
CX
+
,^
X
0
l-l
CX
t4-(
0
4-4
in
in
aj
C
CX
a
0
l-H
CX
+
,_^
X
0
1-4
^
Id
^
Id
•4->
CX
CX
a
0
a
0
l-l
OH
l-l
OH
en
m
4-J
4—>
a
a
CX
0
a
0
l-H
l-H
(X
+
r—H
X
0
l-H
CX
CX
u->
0
a
+
,_^
X
0
l-H
<4-H
0
0
-t-4
4-»
-4-'
03
03
z
z
VO
en
en
13
13
•4-4
4—"
0
0
4->
4-4
l-H
l-l
CJ
03
0
03
ed
z
z
en
en
4—'
4->
in
OJ
^
Id
4—>
CX
a
0
OH
c^
4-4
aj
a
a
0
CJ
4—>
a
<4-l
UIO
a
CX
en
UIO
^
>^
^
ive
CO
ive
aterB
:z;
CO
0
OH
'^
13
0
^
+
iS
0
l-H
CX
«4-l
0
cd
z
Cd
(N
B
in
jj
X
^
Id
0
4—»
l-H
CX
^4-H
0
aj
CX
a
0
OH
l-l
3
03
z
©
l-H
CX
+
^
X
0
13
aj
'•4->
oc
U
O
aj
c«
©
+
, ;
X
0
l-H
VH
a
CX
a
com
0
l-l
aj
u
>
4-^
a
a
l-H
l-H
+
^
X
0
\-l
CX
t4-H
a
+
, ;
X
0
«4-l
0
0
0
•4—'
•4-J
-t-j
4-4
03
cd
03
0
1-H
r-
t-«
<s
t»-
w
Z
fo
^
t^
a
a
CX
+
y—i
X
0
l-l
a
C+-I
03
Z
r-
X
X
W
a
«4-l
0
:2
l-l
l-H
(4-H
z
l-l
+
._;
X
0
4—4
CJ
03
X
w
0
0
-t-4
•4—'
z
VO
t>~
00
r-
r-
t-
13
tH
CN
a
VO
13
0
+
0
X
0
l-l
tH-H
X
0
W
a
l-H
a
aj
>
CO
0
OH
^4-1
0
4-4
iCX
aj
0
•4—>
0
03
3
cd
z
03
:^
^
aj
'4-4
• f-H
CX
4-J
Cd
^
UH
0
=4-1
r-
309
13
l-l
in
H Z
VO
UIOO
a
a
UIO
P^
CX
UIO
cd
UIO
-4-4
Pos
aj
H
VO
ble
<
X
Natl
fl
©
ON
f-
0
00
1-H
00
fS
00
m
X
CQ
m
fN
rN
en
^—^
•t->
^*-.^
^>^
o
O
CJ
sa
^
^4-1
^
C4-I
^
C4-I
aj
aj
aj
13
aj
aj
13
aj
aj
X
X
1-4
^ * i ^
aj
cd
-t—>
0^
S-H
_>
-t—>
CO
O
OH
• ^H
S
4-J
0
^
4—'
1—1
CO
0
OH
03
<4-l
aj
X
CJ
13
aj
X
s
CN
aj
(50
0
>
•
0
•4-1
•*->
aj
s
(N
0
z
4-4
CJ
<N
•4—*
CJ
^
«JH
^
CH
aj
aj
g
0
aj
e^i
r<-,
^—-^
^—^
^-^
CJ
CJ
4—>
0
03
c
f^
5^
OJ
0
X
aj
13
aj
X
X
Z
s s
en
N—.^
to
0
13
en
4 - '
.^
4—•
^
-1—»
CJ
,<^
OJ
Ci^
aj
aj
aj
OJ
CJ
CJ
_>
_>
>
<4-l
4—>
4—"
CO
CO
0
0
OH
OH
<4-H
;-5
>
4.J
• —H
CO
CO
0
0
0-
OH
U
>>
OH
OH
<N
'=t
fN
ture ofp
<
>>
0
0
0
OH
OH
OH
en
in
fN
•^
^
in
a>
C
l-H
03
z
C
^
Id
4.J
z
03
D
£
0
CX
^
Id
4—>
CL
B
(U
X
in
aj
^c
Id
0
4—>
l-H
CL
V H
0
aj
^
•4-^
c
aj
a
a
CJ
CJ
CJ
aj
aj
aj
aj
aj
_>
>
CJ
•*->
•4—>
• 1—.
>
•^
CO
CO
CO
0
0
OH
OH
0
0-
VO
VO
^
VO
^
13
13
13
13
13
1-4
l-l
l-l
l-H
0
0
0
0
0
^
^
^
^
-4—'
•t—>
+
4—<
l-H
0
0
03
X
w
CJ
^
CJ
VO
VO
0
0
0
C3
z
0
CX
Z
cd
<H-H
oble;
CO
ange
•4-4
CO
0
4-*
03
z
•4-"
• ^H
CO
•
0
1 >
4—1
P-I
+
X
•t—>
1-H
X
l-H
^
4—'
•t—1
P-I
l-l
^
B
aj
_>
0
^,, >
^^
m
P-I
o
l-H
0
_>
Prom]
X
13
ture ofp
B
aj
.>
X
CO
ffe
OH
aj
aj
1 >
Exa
0
aj
aj
en
ffe
0
D
D
¥
^—^ ,^-^
^
Exa
O
0^
13
aj
• ^
ffe
CO
;z:
^^
of probl.
CO
Z
0
Avoidane
CO
0
,^
Exa
.—f
1—1
•4—>
• 1-H
0
Prom]
>
<
T*"^
ffe
•
<
T"''*--
H /
ture ofp
_>
'4-'
^^
en
s^ -y
en
^
ffe
>
^^
Prom]
aj
aj
en
^^i_i^
ffe
aj
aj
^ - 1 ^
probl. +
aj
di
CN ^
V- _
isitive CO
0
CJ
of probl.
0<
CJ
CJ
•
Cd
^^
CJ
• 1-H
•f-4
CQ
.^.^
en
\^^
ffe
^ i _ i ^
ffe
H
^_^
^-^
en
ffe
L H
,—,
en
ffe
<
Ratei
xono
s
OH
4—4
03
z
4-4
03
z
B
,
z
- H
d
aj
303
TT
00
.14
CM
€4
in
00
VO
00
r-
00
, ',
a
0
l-l
X
0
,
CX
OH
4-4
4-4
03
03
03
z
00
00
as
0
1-H
as
ON
310
^
^
^
+
aj
w
w
>
X
0
l-l
w
w
VO
ON
r--
CX
-4-4
Z
z
H H ;?;
^
03
z
©
0
^
,
4-4
00
0
xact
03
o
U
X
0
^
X
0
•4—4
a
xact
l-l
;
+
a
+
aj
it. of
<\j
,
0
l-H
CX
0
'^
VO
xact
w
a
+
a
aj
^
VO
4-*
xact
0
0
l-l
a
0
CX
:^
com
^
CX
in
Posi
l-H
CX
a
a
en
ofpr
Natu
ntinu
l-l
4-4
ofpr
aj
en
•4-J
ofpr
13
X
en
0
xact
o
aj
^
a
aj
VO
fpro
,
0
fN
Natu
fpro
eci
a
jj
X
3bl.+
cd
t. of
OJ
•4—4
^
P-I
H
(N
P-I
l-l
ipj
<
X
e4
P-I
©
C3
©
fs
as
f*^
TT
as
as
\n
as
ON
CQ
l-l
aj
03
0^
.^^
^-^
^^
O
o
^*-^
O
^**^
aj
(50
C
03
XCJ
O
Z
D
(50
c
cd
X
o
o
Z
^.^
m
^^
r-,
^^
<N
•t->
•4-J
•4-4
4-J
o
o
o
^
^aj
?t^
C4-H
o
^
tt-l
aj
^
«+x
aj
o
aj
a>
aj
aj
o
_>
13
D
_>
_>
•4—1
4—4
^*-^
aj
bO
c
03
X
Z
^^
•^w^
X
•4—'
CO
O
OH
• 1-H
S
CO
en
^_^
r^
•4—'
4-4
CJ
CJ
^ ^
<
z
^
CH
aj
aj
>
• 1-H
•4—'
CO
CO
CO
O
O
O
OH
OH
OH
<
g
^ ^
^_^
CO
CO
••-H
4—>
o
o
CH
'^
C4-H
aj
aj
aj
aj
^
^
CH
aj
13
aj
• ^H
s
CJ
,'^
''-t-H
CJ
u
>
>
>
•*->
4—1
-t—'
'co
O
CO
O
OH
OH
OH
^_^
^ ,
, ,
CO
CO
r^i
4—»
4-J
-^^
• —H
X
^_^
r-,
^-—H
—^
CO
O
u
>.
Taxono
B
<
l-H
aj
cd
r-v
^ • v .
,^-^
en
^*-^
4-4
o—^
^
o
o
too
,^.^
^ - i ^
CO
CO
^•-^
•4-4
^^^^
en
CJ
CJ
aj
aj
aj
D
aj
O
.>
_>
Z
4-4
'•t-4
• -H
aj
_>
o
Z
X
CJ
O
in
l-l
^«—^
aj
aj
4-J
03
oi
"Id
4-4
CX
a
(N
S.—^
a
^
^*-^
13
o
CX
l-H
<-l-l
o
l-H
aj
X
o
l-l
CX
z
<
>.
o
4-J
03
z
OXBl
B
in
in
ai
4-4
03
_o
Id
•4-4
cd
•t-4
CX
a
l-l
o
OH
o
l-l
OH
c
o
U
H
_>
4->
•4-4
• 1-H
o
CO
Z
CO
^
13
l-l
o
o
_>
_>
4—>
4—>
4—>
• f—1
CO
CO
CO
CO
O
VO
CN
^
Tf
in
CO
^'—^
O
4-4
o
cd
X
s
13
a
X
o
l-l
CX
tM
o
l-H
O
303
^
^
1-4
4-4
13
<N
"^•^^
a
aj
l-H
o
+
, ;
X
O
l-l
CX
<4-l
o
4-J
03
X
O
l-l
a
<+H
o
OJ
1
03
z
^—^
4-4
a>
a
a
o
CJ
aj
_>
• 1-H
VO
13
l-l
CJ
03
X
w
(N
a
aj
X
o
l-l
a
l-l
3
03
z
CN
CX
B
jj
a
o
CX
+
X
o
l-l
O
<4-l
OH
CX
o
^-»^
o
l-H
CX
t4-l
o
aj
z
en
CO
•4-4
4-4
CX
a
a
l-l
o
l-l
o
+
+
X
X
a
o
l-H
a
CX
«4-l
o
o
4.J
03
4-4
ri
o
o
CX
aj
a
a
o
o
aj
_>
•4-4
• 1-H
CO
o
OH
03
z
Tt
O
H Z
311
a
CX
a
l-l
o
1-4
o
+
+
X
X
CX
o
l-l
a
CX
o
l-l
CX
^4-H
t4.H
o
o
•t-j
•4-J
03
ed
Z
Z
in
VO
o
o
o
+•
X
o
o^
Id
4—»
CX
ro
^
•*->
c:
aj
+
aj
X
CJ
>
• 1-H
4—>
O
OH
o
CX
l-l
«4-H
o
4-4
03
z
00
©
ON
o
CX
o
a
o
<-l-4
4-J
03
a
a
o
l-H
l-H
o
l-l
o
X
l-H
OH
O
a
o
CX
q-1
13
CX
+
l-l
^
-t->
a
Z
-t-4
«M
OH
03
-4-4
1-H
4—'
O
4-4
o
o
o
aj
_>
03
r-H
C4-(
£
CO
en
o
l-l
aj
4-4
z
13
OJ
l-H
X
en
a
•4—"
-t-4
<4-H
o
aj
en
^*-^
•t-4
CO
Z
o
cd
Z
ON
ON
>
4—>
OH
z
ON
_>
OH
+
CX
aj
aj
O
4-4
l-l
OJ
aj
O
o
o
aj
aj
OH
o
X
aj
aj
O
03
a o
CJ
CJ
OH
•t-<
00
03
.>
u
g
O
o
'•4->
X
X03
•4—'
<
OH
l-l
oi
13
aj
:3
aj
aj
X
<
aj
(50
c
«+1
W
B
©
CO
O
O
O
z
^^
O
OH
l-l
(4-4
03
CJ
^^
OH
13
+
^
CO
4—>
<
OH
1-4
X
o
OH
CO
o
OH
m
•4-"
03
03
X
o
CO
^-^
aj
aj
aj
•4—'
^-^
m
^^^
4-4
" ^ H - ^
-t-4
03
z
CO
en
4—>
a
a
l-l
o
l-l
o
CX
CX
+
X
o
l-l
a
<+H
X
o
l-l
a
^4-1
o
o
•t-4
•t-i
03
03
z
z
o
1-H
1-H
OQ
l-H
aj
03
0^
G_;
1—1
<+H
fS
a j Q
too +
<
z
CO
4—>
0
<
z
<
Z
^
M-H
'tobO
aj
_>
o
X
4—>
aj
S
CO
>
0
OH
4-4
CJ
aj
13
aj
aj
l-H
4—"
^
t4-l
a>
c
rT
en
^
"cj
,^-J
C^
CJ
r<-,
•4-4
CJ
g
aj
>
>
CO
0
0
i
OH
OH
CO
CO
CO
<
4—>
4—'
-t—>
CJ
CJ
CJ
Z
03
(50
0
X
•*-•
CO
r^,
^^
•4—>
CJ
• 1—1
CO
CN"
c^
<->-,
aj
-0
^
«4-l
aj
aj
• 1-H
4—>
<
CJ
,<^
'-H-^
"cj
,CJ
<
g
CO
'—'
CJ
,'•->
'~-t—H
' - 1 —
CJ
CJ
CJ
CJ
CJ
CJ
>
[^
>
>
;-5
C
0-
OH
^
CO
cT
jj
/-»
X
0
u
'^••—t
; • =
Tr.
0
--•
0,
aj
:^
U
?^.
OH
OH
OH
<N
^
CO
l-H
CX
«4-H
0
4-4
03
z
l-l
aj
l-H
CO
CO
3
0
0
OH
OH
a
z
03
'^
•t—4
a
a
0
0
_>
l-H
-4—)
aj
!-l4
a
0
l-l
OH
^
^
-t-4
4-4
c
aj
C
a
a
0
CJ
aj
>
• 1-H
aj
a
a
0
CJ
aj
_>
'4-4
en
4—'
CX
<s
ri
-^
in
VO
+
, '
X
0
t^
/—I
03
«
ed
z
^
^3
+
OH
z
aj
-1-4
l-l
cd
d
0
l-H
-4—«
o
U
«4-l
^
H H Z
312
0
CX
Z
00
.^
•4—•
pro
-*—•
•4—*
pro
4—>
n~,
pro
•4—>
—
pro
r^*
+
0
+
+
+
0
+
X
0
l-H
a
ti-i
0
CJ
aj
>
• l-H
4—4
• —H
CO
0
OH
•4-4
03
4-4
CX
u
CX
X
0
\-*
CX
X
0
l-l
CX
t4-l
<4-l
0
0
0
•t-j
4-*
CX
tH-l
ed
X
0
i-
CO
0
OH
'cd
en
CO
en
•t-4
4-4
-4-J
4—>
a
C
a
a
VH
l-H
CX
+
,__^
X
aj
,_^
X
>
•4-4
4-4
0
4—"
03
z
0
fs
CX
+a
, \
X
0
4-4
Cd
0
ed
4-J
CX
cx
^4-1
4-4
^
c
a>
1—
>
• l-H
4—>
Z
+
as
0
Z
0
CJ
0
Cd
aj
z
z
CO
X
0
l-H
c
z
^
en
4—>
CX
+
0
CJ
+
^-^
X
0
aj
, \
a
_>
4—>
UIO
a>
m
CX
ro
OUI
C
0
aj
Natu
0
X
0
0
0-
UIO
X
O
l-l
4-4
<-l-H
c/j
S
OH
UIO
a
aj
•t—'
* 1-H
- a
• ; : ^
X
0
. ofpr
0
CJ
0
>
X
Posi
X
CJ
CO
1)
aj
. ofpr
OH
0
a
a
aj
CJ
>
• —H
-4-J
• —H
. ofpr
o
0
aj
4-J
13
CO
. ofpr
CO
+
c
a
Posi
•4-4
• 1-H
aj
-4—»
—
aj
pro
0
-4—»
•4-J
<N
CT
0
aj
Avoidane
OH
+ word 1
a
a
0
0
C4-I
ntinuec
0
ive
aj
^4-1
CX
•
CO
0
C'
^4-1
^
<4-(
CJ
UIO
oi
CO
0
UIO
cd
CO
0
CO
CX
•^->
CO
UIO
aj
•t-4
H
4—"
. ofpr
l-H
4—'
. ofpr
<
c<
'•t-4
Posi
C3
©
X
>
0
C^
tl-l
aj
UIUI
c
:z:
a
©
_>
4—•
dui
<
S^
_>
g
dui
a
_>
0
dui
o
l-H
aj
aj
4—"
4—4
OAl
X
CT
^
U-i
-4-J
"cj
ro
UIUI
+
aj
aj
i'
<
dui
B
^
^4-1
aj
aj
UIUI
CX
^
^4-4
dui
oi
4-4
pro
erB
s
03
g
aj
<i>
X
•*->
z
<
^
C4-I
aj
13
<U
CO
0
<
at. of robl.
^
t4-H
4—>
robl
g
CO
Posi
0^
CJ
<
ive
03
4—<
Posi
aj
•4—>
M
CN"
l-H
pro
<
tt alone i
axono
a
4—>
•4—>
03
z
03
z
z
z
Cd
^H
fS
<^
Tf
in
<s
r<
fS
«s
r\
^"
^"
OH
aj
aj
aj
aj
g
CO
<
4-J
Z
03
ffec
0
too
aj
<
0
OJ
aj
X
itivi
O
z
ffec
CO
0
^—^
itiv
u
CO
OH
X
a
CO
^*-^
4-4
ffec
aj
o"
CO
^*-^
4-4
itiv
aj
aj
z
o"
^"^-^
<L>
ang(
oi
<
•t-4
ffec
03
CO
• ^ • H ^
•4—>
itiv
aj
•4-4
CO
^—.^
ffec
l-l
itiv
<
CO
CO
0
0
0
OH
OH
OH
^
o"
0
Z
CO
aj
aj
aj
aj
,>
'4-j
»1—1
CO
g
en
^^,
1 ^
aj
aj
>
>
4->
4-4
• 1-H
• 1-H
CO
CO
0
0
0
OH
OH
OH
l-l
CX
0
en
X
<
^
T"***-
\^ ^
Z
>
bO
1^
bO
'co
0
0
u
CJ
aj
0
z
• 1-H
4-4
aj
CJ
. eff.
<
^
ffect
l
ange
_
oblem
CO
^
1 ^
ffect
H
CO
^ 1
ffect
i_
ffect
<:
Rate]
xono
u
>>
s
l-H
OH
aj
3
'0
•
a
0
CX
0
l-H
CX
+
+
l-l
0
o3
•t-<
CX
a
0
VH
aj
z
en
en
CO
4-4
4-4
4-4
0
1-4
CX
+
CX
C:
CX
a
0
a
0
CX
CX
C
JH
C
+
0
l-l
l-l
+
aj
0
X
0
l-l
CX
X
0
l-l
CX
X
0
l-H
CX
«4-H
tl-(
«4-l
0
0
0
•t-4
4-J
4-J
03
z
4-4
4-J
4-*
Z
z
z
Z
z
en
en
en
CO
en
4—4
4-4
•t-4
•1-4
4-4
4—4
a
a
a
C
OAl
0
0
OH
03
+
r—4
«4-l
CO
l-H
CX
,__4
0
4-4
a
0
F—4
<+H
OH
en
03
03
—
•t-4
X
0
l-H
CX
03
B
a
a
0
1-H
03
tiO
^_^
X
0
1-4
CX
^-H
^
>%
aj
one 1
erB
Pi
CX
in
dui
03
4-4
dui
4-4
•t-4
^
>
•t->
>
<
ro
Z
aj
03
CO
<
7*"--.
>
• 1-H
4-4
'co
0
OH
,__^
X
0
l-H
CX
«4-l
0
4—>
c
aj
a
a
0
0
ai
>
•4-'
'co
0
OH
03
©
l-l
CX
+
aj
'•4->
c
o
U
©
+
alon(
«4-(
C4-(
0
0
4-J
4-i
Z
Z
O
rt
a
0
•t-j
l-l
OH
VO
h-
<s
<S
00
fS
l-l
+
^
X
0
l-l
CX
^
X
0
l-l
C4-H
«4-l
0
03
l-l
CX
X
0
l-H
CX
Z
ai
.>
l-l
CX
^ ;
X
0
l-H
CX
03
aj
0
CJ
•t-4
Pos
13
+•
a
a
en
aj
+
a
t4-H
0
0
4-J
•t-4
4-J
03
03
0
aj
_>
'•4-4
• —H
CO
0
OH
z
ON
0
1-H
rq
rq
m
m
m
H Z
313
a
+
, ;
X
0
IH
CX
tH-H
0
4-4
C
a
0
0
aj
_>
"4-4
• 1-H
CO
0
OH
4-»
Cd
z
CX
l-H
0
, ;
X
0
l-l
4.J
OUI
l-l
CX
c
411
UIO
a
0
CX
OUI
0^
a
UIO
aj
cd
•t-4
UIO
H
4-4
in
l-l
UIO
<
UIO
axo
ea
03
z
m
f^
rr
fn
in
m
APPENDIX P
CLASSIFICATION OF COMMENTS ACCORDE^G TO
SUBSTANCE AND STUDENTS' TEXTUAL
REVISIONS FOR THE FIRST EXCHANGE
314
n
hH
•*J
<
z
<
z
<A
CO
•t-4
CJ
en
aj
4—>
too
C
03
too
C
o
_>
Z
4—'
x:
o
'co
O
13
OH
o'
aj
tJQ
C
03
CJ
'^
X
o
aj
aj
:3
X
0)
o"
cS
(D
aj
aj
'co
O
<
<
g
Z
g
CO
CO
CO
4—"
"cj
CJ
,CJ
"cj
/^
'•J
X
aj
CJ
_>
4—'
<
_>
>
>
O
Z
OH
CO
CO
O
O
o
CO
OH
OH
OH
CO
CO
CO
u
•t—4
(^
>>
o
ouo
B
C3
O
- 1-H
•t—4
<
X
C^
O
.
<
g
H
1-H
<
<
g g
CO
•t-*
0^
CO
<
g
CO
S'
<
<
<
•t-4
d)
Z
Z
Z
CJ
too
c
aj
aj
CI
aj
aj
03
X
o
_>
13
o"
u
•t—4
o
'co
O
Z
OH
4—'
'co
O
bO
C
03
4—>
CJ
• ^ J
"cj
,aj
aj
aj
X
o
O
a;
>
_>
4—'
• -H
Z
CO
O
OH
OH
•
1—1
4-J
CO
CJ
OJ
OJ
>
4—»
CO
O
o
OH
OH
in
^^^
^*^
d
4—4
c«
X3
:3
CO
OQ
n
;H
o
c:
o
• ^ ^
<
g
CO
^—^
^**-^
b
Jd
•t—'
aj
4-4
c
o
U
'^
^—^
i3
3
X
a
a
m
^
CO
^H-^
^*-^
*"*—^
a
a03
^
^
3
X
l-l
03
,2
3
X
l-H
03
03
03
03
l-l
o
o
o
o
>O
>
CO
CO
CO
CO
>o
'
z
CJ
^—^
•1-4
-^
l-H
c
(a
c
o
U
c
o
U
—
—
•4-4
•t-4
•t-4
03
,
^
^-^
03
aj
4—>
O
CO
CJ
a
a
'B
03
VH
03
o
aj
•4—'
X
O
aj
c
o
U
in
--
C3 5P
<
>.
B
©
g ^
©
X
C4
•OS
<
VO
<
H
Z
o U
c: >.
VO
<
g
CO
^:
^
X
3
03
o
^ §
^
3
X
3
03
03
CJ
X
03
CJ
b
3
X
03
CJ
O
^
Jd
X
3
03
>
>
>
>
f*^
Tt
in
VO
r-
o
<
g
aj
aj
4-4
c
o
o
CJ
O
>
o
.2 a
^
VO
o
in
CJ
CO
CJ
03
cd
CO
'a
X
o
aj
4—<
aj
4—>
CJ
CJ
c:
o
CJ
X
u
u
ON
o
^ H
r<
f^
^ H
^ H
^ H
1-H
s s
^ H
^ --5
cd
o
1-H
C4
r^
©
H Z\
315
00
CQ
l-l
aj
4-J
Cd
oi
^^
—^
•^H^
^^
CO
^*-^
4-J
^"
a>
o
t+H
too
X
too
aj
l-l
o
aj
aj
aj
_>
'-t-4
CO
O
OH
^-^
^^
,,~^
^—^
^H-^
'**-^
CO
-t-4
CO
-t—>
o
o
aj
aj
aj
>
_>
^
«a j
4-J
CO
O
OH
en
^.^
^.^
^ • * — ^
^
CO
-1-4
-t-4
0
0
t+H
aj
aj
>
^
«a j
• 1-H
4—'
• 1—1
CO
CO
CO
0
OH
^^
(N
^
^..^
en
4—4
-4—4
CJ
CJ
CJ
aj
aj
aj
•i->
qH
aj
aj
>
• 1—1
4-4
0
•
aj
'4-4
OH
CO
>
• l-H
4—'
CO
0
0
OH
OH
T3
aj
X
s
_>
^^^
en
4—'
0
<
Z^
Z
cx3
aj
aj
^^
CO
•t—>
0
g
>
>
4—*
• —H
-4—*
• I-H
'co
0
CO
CO
0
^^
4—>
CJ
'O
aj
!«i
S
0
OH
OH
^,
<N
aj
aj
aj
•4—4
OH
<
>
• 1—1
-t-4
03
too
aj
z
S
©
<
Tax
©
,—^ ^.^
CO
CO
,—^ ^-^
CO
CO
,—^ ^-^
CO
CO
,—^ , ^
CO
0
^ ^ ^.^
CO
1—1
aj
•t-4
03
oi
o
CJ
aj
aj
aj
aj
aj
>
>
>
0
0
CJ
CJ
aj
aj
aj
aj
aj
aj
aj
aj
>
>
>
>
aj
4-4
0
CO
CO
OH
OH
O
O
a>
CO
O
OH
CO
0
OH
CO
CO
0
0
OH
OH
CO
XCJ
0
CO
aj
4-4
03
oi
^
03
3
X
03
CJ
O
>
^
^
a
a
03
l-H
o
CO
'^^^
^
,2
3
CJ
O
^
l-l
03
a
a03
l-H
0
>
CO
^^-^
b
03
3
0
0
>
'^
^•^-^
^
^
^—^
;J:^
CO
C
a>
c
OH
0
'4-4
• 1-H
fe
03
a
a
03
VH
l-l
S
0
0
4-4
aj
4-4
0
U
^*-^
CO
l-H
0
• 1-H
c
03
X
aj
-4-4
0
aj
aj
>
• ^ 1
cd
03
too
too
aj
aj
z
^^
^—^
-t-4
^
^*-^
l-H
03
aj
aj
4—>
4—4
0
0
>
C3
U
c
l-H
l-l
• .—t
4-4
4-4
too
aj
0
aj
^—^
.
toil
too
too
aj
in
^—^
g
OJ
z
^^m^
.—I
too
;^
03
toiO
aj
l-l
^—.
M-H
>
PQ
<
aj
0
OH
1—1
too
'^
OJ
aj
0
z
^,
aj
C
03
<
g
l-l
C
0
U
z
<
Z
a
a03
^
^—^
•t-4
c
aj
4—>
0
U
l-H
0
^
>-»
S
©
Taxo
cs
<
aj
J3
-t->
03
riS
3
a
acd
t_i
n3
CO
CO
CO
l-l
o
>
cS
Tf
in
l-l
^
3
X
03
CJ
03
a
a03
l-l
O
0
VO
r-
03
3
X
03
CJ
^
in
fe
-t-4
a
a03
aj
cd
a
a03
l-H
1-4
l-l
4-4
0
U
0
0
0
00
ON
0
1-H
r<
<s
CO
0
303
X
CJ
aj
VO
4-4
-t-4
l-l
03
aj
OJ
a
a03
<
z
-t-4
-4-4
0
0
u
u
m
TT
in
VO
n
«s
<s
-t—>
c
aj
c
•4—'
0
l-l
0
U
>
>
aj
'-I-4
co
U
OH
aj
I
e«
©
H Z
316
n
n
rq
r<s
Rater B
CO
o
CJ
_>
•t-j
CO
aj
tiO
03
c
03
o
o
X
o
o
C
^M-4
aj
aj
o
aj
too
•t—>
X
Z
Z
CO
CO
CJ
^q - i
o
^
aj
aj
aj
aj
>
_>
_>
4-'
•4—'
• —H
4-4
• 1-H
CO
0
^
a>
«4-H
aj
CO
'co
0
OH
OH
CO
CO
0
^H-^
^—^
^H-^
O
O
OH
C4-H
CO
OH
0
CO
en
CO
CO
tiO
0
0
0
0
03
5^
<H
aj
^t+H
^
yx
aj
^
t+H
0
aj
aj
aj
0
_>
• -H
X
Z
0
>
4—4
aj
_>
4—4
aj
aj
>
• -H
4—'
CO
CO
0
^
U^
aj
aj
>
CO
CN
0
CJ
^C H
^
C4-H
OJ
aj
<u
_>
TS
0
'•4-4
X
0
'co
0
s
OH
OH
4—>
• -H
'co
0
'co
0
CO
CO
0
0
OH
OH
OH
OH
^
CO
CO
CO
CO
U
>^
a©
©
X
et
H
<
l-H
aj
CO
^—^
4-4
4—4
03
Pi
en
o
'^^.•^
aj
too
^'•H-^
•t-4
q-H
q-l
aj
aj
aj
a>
_>
4-J
>
• l-H
4-4
• 1-H
CO
CO
O
O
OH
03
X
o
o
Z
•t-4
CJ
«4-l
aj
aj
_>
q-H
aj
03
OJ
aj
X
>
z
4—>
• 1-H
4-4
• 1-H
CO
CO
O
0
OH
OH
4-4
CJ
0
en
^—^
4-4
aj
a>
.>
•t-"
CO
0
OH
OH
4-4
•t—'
4-'
0
0
0
tM
^q-H
«4-H
q-H
C4-H
aj
aj
OJ
aj
aj
aj
aj
aj
aj
aj
>
_>
_>
4—»
•4—1
4—>
CJ
0
«4-H
CO
•N—^
X
0
1-
CX
<-l-l
0
aj
CJ
C
cd
>
4—1
• 1-H
4-4
CO
CO
CO
0
0
0
'co
0
4—<
CJ
<
g
>
;-5
CO
0
OH
OH
OH
OH
OH
^
^
^
CO
in
T3
•
1-H
0
>
<
PQ
^
•
^
-
•
^
-
^
in
^
CT
in
^
l-H
aj
-t—4
03
l-l
03
£
c
03
o
l-l
4-4
C3
aj
4-4
c
aj
•t-4
4-4
U
U
c
o
G
O
1-4
03
a
a03
o
l-l
CO
CJ
• 1-H
c
03
X
0
aj
•4-'
c
a>
4—>
$=l
•t—>
c
ai
•t-4
0
c
0
U
U
CO
0
303
X0
aj
l-H
4—>
Cd
C
aj
a
a
03
l-l
0
4-4
c
0
U
•*->
C
aj
-t-4
c
0
U
^
Jd
3
X03
CO
0
'B
a
X0
0
aj
0
2
>
•*-»
c
aj
c
0
4—"
U
<i:
>^
B
©
at
H
<
aj
cd
0^
-t-4
cd
3ed
X
o
-o
aj
CO
CO
•n
in
1-4
aj
4-4
CO
D
• 1-H
•t-4
•1-4
c
o
c
o
U
U
03
0
c
03
303
XCJ
O
aj
X
o
o
>
>
•1-4
CO
CJ
aj
4-4
C
aj
•t-4
c
0
0
U
U
S
03
C
03
XCJ
ry(3:
e:
ry(3;
©
•t-4
aj
•1-4
aj
03
3
X03
4-4
X
C
0
c:
0
U
U
303
0
0
S
>
Tt
in
VO
m
m
m
0
0
in
CO
CJ
• l-H
C
03
X
CJ
aj
>
S
00
ON
0
m
m
TT
4-4
c
aj
•t-4
c
0
U
:3
c
• I-H
c
o
u
T—1
n'i
aj
^
H
,:4
a ©
00
ON
<s
<s
o
m
1-H
m
<s
m
m
m
H ^
317
rf^
^
'^
k-
aj
3
CO
CO
o
CO
^—^
•4—>
^—^
•4-4
^—^
^—^
OJ
too
c
0^
aj
y
aj
OJ
>
>
."CO5
O
.-5
CO
• ^ J
CJ
Cd
^-^
—H
too
c
CJ
OJ
Cd
OJ
OJ
CJ
O
O-
g
,aj
X
o
Z
r^i
^—^
CJ
"oj
OJ
O
>
X
>
Z
!-E
CO
r ^ i
g
OJ
^-^
^_^
._i
CJ
c,
^..^
r^,
._
,„J
r ^ i
^^^
._
too
c
OJ
,0J
CJ
CJ
,0J
CJ
,0J
f—
r-
CJ
CJ
"oj
OJ
"oj
OJ
13
3
O
^—
>
>
>
>
•-2
too
c
cd
Z
^'.O-E
c
^—^
o
z
' • ^
!o
w
OJ
' • ^
\r.
!o
*—
^
OJ
X
^
^
u
>.
X
<
CO
CO
<
CO
aj
^—^
•4-J
^—^
^—^
-^H
03
CJ
CJ
Z
aj
OJ
OJ
OJ
>
>
l-H
e4
H
•—H
CJ
o^
<
g
P^
PQ
'.O
c/2
O
^
o
o.
^
'sT
^H—^
CJ
cd
X
4—4
Z
o"
g
1
1
^_
ro
•4—4
, ^ j
r ^ i
^_^
,.
,
•
OJ
CJ
,0J
CJ
,0J
aj
OJ
3OJ
3OJ
3OJ
O
>
>
>
>
CO
CO
v:
O
^
(N
r-,
OJ
o
OH
^
CO
Z
CO
O
<
X
o
>
OH
^_
Z
OJ
o
O
CO
<
• ^ J
too
c
"oj
aj
CO
^^^
o
ange
ouo
E
_^
,
^
,
^
X
^
ri
'=1-
'^t
V.
v.,
cd
l-l
-t-j
03
Oi
l-H
VH
Cd
Cd
a
a
cd
E
E
cd
l-l
VH
o
a
CO
CO
•t—4
c
aj
c
o
O
-t—4
VH
03
a
a
ed
VH
-t-4
-t-4
•*->
c
aj
-t-4
c
o
U
c
aj
c
aj
-t-j
o
U
4—'
o
U
o
o
3
'E
Orga
aj
.^_(
c
^
3
r
>
.4.^
VH
C
Cd
^
a
o
^
3
a
o
E
cd
Vr 1^
,^_»
c
OJ
3
O
Cd
f—
n
E
E
Cd
V.
cd
^^
O
V.
V.
Cd
cd
c:
p
E
r-
V-
<
>,
OXBl
c
<
V.
aj
cd
—H
Jd
Jd
3
X
3
X
03
CJ
O
>
03
o
•t—>
V.
c
aj
c
o
U
cd
•4—>
o
E
E
cd
V.
'' ^
^
OJ
3OJ
3
3
3o
o
-4—'
c
o
U
tion (2;
B
©
cd
'5
OJ
3o
cd
OJ
E
E
3
cd
••
03
3
OJ
3
V-
cd
r-
^
cd
V.
^
V.
•'
^
^^
^.z
r^
Tf
IT)
in
ITj
IT*
"^
bO
>
V.
VH
o
CJ
C
fS
X
cd
TT
c«
©
f*^
Tt
-^
-"I-
in
rt
VO
• ^
r-
"^
00
"<1-
H Z
318
ON
Tt
o
IT)
^
in
fs
in
<
CO
CO
en
en
(N
CO
CN
CO
(N
CJ
CJ
CJ
CJ
0
0
CJ
CJ
0
0
CO
CO
CO
r^,
0
0
0
0
aj
aj
aj
_>
_>
_>
TD
'•4-4
•t-4
4-J
'•t-4
aj
CO
CO
CO
CO
o
OH
O
o
OH
o
OH
CO
'^^y
OH
>
X
» 1-H
4-4
S
'co
0
• ^H
aj
X
%
OH
_>
4—'
aj
CO
CO
^•.z
^-^
-t-4
aj
aj
aj
aj
>
>
_>
'co
0
CO
0
>
z
%
aj
aj
OJ
aj
'co
0
aj
aj
X
13
X
ffe
aj
aj
ffe
ffe
aj
TS
ffe
ffe
aj
aj
_>
a>
a>
ffe
ffe
aj
aj
ange
ffe
aj
aj
ffe
ffe
oi
ffe
cd
ffe
aj
ffe
VH
CO
0
OH
OH
• -H
4-H
OH
4—'
0
CO
0
OH
OH
0
<N
U
>»
CO
CO
CO
VH
^H-Z
^•—HZ
aj
•t-4
•1-4
^• t .^J
o
^
u-t
cd
oi
^
CH-H
o
^
q-H
aj
aj
aj
aj
o
erB
03
0^
•t-4
4—"
CJ
CJ
#
C4-I
aj
aj
en
''^•^^
4-"
4-4
CJ
0
0
0
^
C4-I
^
(4-1
^
C4-I
^
(4-1
^
54-1
aj
aj
aj
aj
aj
aj
aj
aj
aj
aj
>
>
>
>
>
>
>
>
>
•t—4
• 1-H
-1-4
•t-4
•1-4
• 1-H
•4—>
• 1-H
•1-4
4-4
* 1-H
•t—>
4—>
CO
CO
CO
CO
CO
-t->
aj
aj
CO
^^-z
CO
OO
CO
z
'^
(N
4—4
^—^
aj
tiO
^
JJ
X
0
^
C4-1
VH
'O
CJ
X
z
CX
C4-1
CO
0
aj
0
aj
aj
03
^^.z
-4-"
CJ
^
U-i
X
aj
0
^3
D
%
X
s
O
O
O
0
0
0
0
0
0
OH
OH
OH
OH
OH
OH
OH
OH
OH
in
'^
in
fN
^
in
^
4-4
4->
4—>
-t—>
-t—'
4—>
4—>
VH
C
c
aj
C
c
aj
c
0
U
c
aj
c
0
U
c
aj
c
aj
c
0
U
Cd
C/5
o
'B
a
X
CJ
aj
VH
Cd
a
a
cd
o
VH
CO
o
• 1-H
Xfe
o
aj
Cd
_N
'S
4—>
C
CO
0
• 1-H
4-4
C
0
U
03
X
0
aj
aj
"^
aj
4-4
4-4
-t—4
C
C
C
U
U
U
0
0
0
4-"
4-*
4—>
C
0
U
4—>
a
ram
4-4
•t-4
CO
^—z
Avoidane
H
<
tiom
X
e4
Orgai
ono
B
0
<
>^
B
0^
cd
Mec]
_C1
03
o
CO
o
0
3
'•t-4
X
cd
03
N
aj
•t-4
0
U
CO
0
1
c
c
•t-4
-t-i
aj
aj
C2
aj
-4-4
c
c
c
c
-1-4
4-4
4-4
4-4
aj
aj
G
0
0
0
0
0
u
U
U
U
u
1-H
<s
m
Tf
VO
ir»
VO
VO
VO
X
aj
ry(3;
fe
aj
03
3
X
0
0
0
U
r-
00
VO
Voca
CO
• 1-H
rgani
03
Mec]
aj
4-4
in
Mec
in
in
VH
nt(i:
<
UIUIl
Taxo
o
0
-13
aj
3
C
•t-4
c
o
U
— H
OH"
VO
ITi
aj
1
H H
CM
es
t-
00
ON
in
in
in
0
VO
VO
©
Z
319
VO
VO
vo
ON
VO
PQ
VH
,^^
(N
r-v
CO
'^—^
^ - 1 ^
aj
4-J
-4—>
03
o
o
<
g
^^
rsi
^-v
..-^
^^
^.^
CO
CO
^—^
CO
^—H^
^—.^
^_^
fN
^^
CO
-t-4
4—'
4—4
-t-'
4—4
•t-4
4—4
CJ
a
o
o
CJ
CJ
CJ
^*-^
aj
aj
aj
aj
aj
^
«a j
^3
aj
aj
aj
aj
aj
>
13
aj
>
_>
0^
• ^
.
X
4—"
• 1-H
CO
• 1—1
4—>
• —H
CO
CO
O
O
OH
• 4
JC"
_>
aj
D
>
CO
• 1-H
•t-4
• 1-H
CO
o
4-4
CO
<
g
^ ^
^^
^^
CN
fN
CO
4-^
J •
o
OJ
CJ
4—*
aj
«4X
aj
OX
aj
aj
^
«a j
13
aj
13
TS
aj
aj
aj
>
X
X
4—>
5:3
aj
X
_>
4—>
• 1-H
CO
CO
OH
O
OH
O
OH
O
OH
OH
CO
CO
CO
CO
CO
<
-t—4
Z
•v
ono
u
>>
s
<
X
et
H
VH
a>
03
oT
rT
^ • H - ^
^•H-^
-t-4
-l-J
o
0^
aj
<
z
C4X
aj
13
aj
13
X
X
aj
too
d
<
g
03
X
o
aj
^ w ^
4-4
CO
-4-*
4—'
4-4
4-4
4—4
o
CJ
OJ
OJ
aj
u
aj
aj
aj
aj
aj
aj
aj
aj
>
.>
a>
a>
aj
aj
4—4
•1-4
• 1-H
>
• 1-H
•t-4
CO
'co
CO
o
o
OH
O
'co
O
OH
OH
OH
^
^_^
,
M-H
aj
• 1—1
^4-4
%
CO
CJ
.>
o
Z
• 1-H
S
o"
^—^
.>
>
4—>
• 1-H
4—<
• -H
• 1-H
>
CO
CO
CO
O
O
O
OH
OH
OH
• 1-H
4-4
too
1^:5
too
aj
l-l
O
aj
>
• l-H
4—»
03
too
aj
z
•' It
PQ
VH
OJ
4-4
fN
^H^^
^H-^
-1-4
-t-J
03
Pi
-t-4
c
o
U
aj
^—^
o
-4-4
03
C
o
U
N
3
i n
i n
i n
^H-^
^—^
4-4
CO
to
CJ
^—*^
aj
• 1—1
^—»^
^—.^
4->
aj
-t-4
-t-4
c
o
U
c
o
U
03
too
o
03
X
CJ
aj
CO
o
303
• 1-H
X
X
o
aj
S
S
in
in
03
o
^^-^
-t-4
-t-"
c
o
U
aj
S
i n
CO
OJ
g
'^
VH
03
a
a
cd
303
X
OJ
VH
aj
s
o
VH
o
CO
CO
^
Jd
^
3
X
3
X
03
o
o
>
03
03
OJ
O
>
<
>^
B
Taxo
©
C3
<
VO
m
VO
CO
CO
Jd
Jd
3
X
3
X
VH
aj
cd
1*
-1-4
-t-4
4-4
-t-4
CO
3cd
<
aj
aj
-t-4
-t-4
c
o
c
o
U
u
z
aj
aj
4-4
4-'
c
o
U
c
o
U
13
aj
o
X
CJ
aj
CO
o
303
X
o
aj
S
S
in
t^
VO
fe
a
a
03
VH
o
4-4
CO
CJ
aj
303
-t-4
c
o
U
<
g
X
VH
Cd
a
a
03
OJ
aj
VH
a
S
03
o
o
>
cd
o
o
>
'-4-4
(=1
o
u
OH
OJ
o
r-~
et
o
1-H
f-
«s
r-
f^
r-
Tt
t^
r*
H Z
320
r^
t^
00
ON
r-
r-
o
00
1-H
00
<s
00
f^
00
PQ
VH
aj
CO
(N
^—^
^'4—
^
-4
o
<x3
-t-4
cd
aj
aj
>
4—>
CO
aj
(N
^
•
—
CO
(N
^H-^
^H-.'
4-4
•t—>
O
c^
aj
aj
13
aj
X
X
13
^
-t—'
• 1-H
aj
aj
>
4—>
CO
S
S
^-v
CO
^CO
—^
^-v,
^-^
^^
'^^
"^^
o
OJ
o
O
O
OH
O
OH
aj
13
aj
X
o
^•-^
aj
too
c
X03
o
o
Z
S
fN
(N
4-J
4-4
o
g
aj
tiO
c
aj
aj
13
aj
13
aj
X
X
03
• l-H
S
S
CO
CO
4—'
4—>
o
0
CO
CO
v^^
s.*^
, 4 _ *
.^^
OJ
CJ
,0J
3a j
>
3a j
>
cE
X
o
aj
aj
aj
aj
o
Z
_>
_>
4—>
•t—>
• -H
CO
CO
CO
O
0
0
0
OH
OH
OH
OH
^—^
_.
^ CO
,—, ^^.^
CO
CO
•4—'
4—>
CO
U
>->
sO
s
<
©
X
et
VH
aj
-t-4
H
cd
oi
^—s
CO
aj
aj
>
aj
aj
>
aj
aj
>
• 1.H
4-4
• 1-H
CO
CO
CO
O
O
O
OH
OH
OH
• l-H
-t-4
,^.^
ro
^^
o
aj
^
«a j
13
aj
X
•4-4
cS
• 1-H
-4—1
(N
aj
>
• ^H
S
^-^
^...^
CO
CO
<
<
^^
g
z
^
o
OJ
^.^
o
too
¥
03
X
aj
^
«aa jj
aj
aj
>
.>
o
Z
• 1-H
4-4
• 1-H
X
o
-t—>
• 1-H
CO
CO
CO
O
O
O
OH
OH
OH
•^
-i—>
4—>
OJ
0
o
aj
aj
aj
0
C4-I
•t—>
-t—>
CJ
aj
VH
CX
0
aj
CJ
C
yx
>
>
4—'
aj
aj
>
•t—'
CO
CO
0
0
CO
0
OH
OH
OH
'^l-
'^
03
13
0
>
<
PQ
^
OJ
4-J
03
0^
^
^ • ^
VH
'si-
^
^ • H ^
•s—^
•*->
VH
-t-4
VH
VH
VH
03
c
aj
03
03
03
a
a03
o
a
a03
o
a
a
03
o
u
a
a03
o
fe
VH
4-4
o
VH
1—1
s^.-^
VH
VH
c
•4-4
o
u
'^
^*—^.
'^
^H-^
VH
VH
03
03
a
acd
o
a
a03
o
VH
in
g
s«-^
VH
VH
OJ
03
03
03
03
303
a
acd
a
a03
a
a03
a
a03
0
0
VH
s*-^
-sT
^H-^
VH
CO
X
o
aj
VH
^s—^
VH
^•H-.^
VH
VH
VH
0
0
^
>^.
E
0^
a
a
03
o
aj
o
U
Tf
00
in
00
-t->
VH
E
03
o
a
a
03
o
VO
00
00
03
VH
VH
fe
a
a
03
o
VH
03
aj
4-J
o
U
VH
VO
<
a
a
03
o
a
a
03
o
Z
o
ON
1-H
rq
ON
ON
VH
in
CO
CJ
• 1-H
03
X
o
aj
VH
Jd
3
X0
03
0
VH
03
a
a
cd
0
VH
>
ryO:
4-J
-t-4
ry(3:
aj
cd
H
'sf
VH
ry(3;
VH
ar (4;
<
et
nt(l^
8
©
ar(4]
©
Jd
03
3
3
X
CJ
03
0
X
cd
0
>
>0
VO
ON
ON
13
aj
:3
c
c
o
U
-t—>
1 - H
O-i
aj
303
.1^
V3
e«
r-
00
00
ON
00
O
H H Z
321
m
ON
Tf
ON
in
ON
r-
PQ
VH
aj
ed
Oi
o"
o"
^—
^
aj
^^-*^
aj
fcJQ
tiO
c
X03
o
o
Z
03
X
o"
CO
"^H-^
aj
"^^^
4—"
o
aj
aj
^H-^
too
CO
CO
<
^^-^
-t-*
s^./
4-4
o
o
o
g
aj
13
aj
aj
O
o
o
Z
>
• 1-H
4—>
X
• 1-H
-t-J
CO
O
aj
%
OH
CO
(N
<
CO
ro
4—4
4—'
, ^ j
o
o
Z
4—1
aj
aj
aj
aj
• —H
>
_>
_4->
4—<
00
CO
4—'
^
«a j
03
X
OJ
Z
f^T
>
o
O
O
OH
OH
OH
CO
O
13
aj
X
CO
OJ
CJ
.,^
CJ
^
«a j
,<u
,«->
3
3
aj
aj
aj
>
>
_>
s
4—»
• —H
OH
OH
OH
CO
CO
CO
CO
•t—4
4—4
4—'
CO
O
4—"
CO
O
CO
0
u
>%
ouo
E
X
et
H
<
VH
aj
CO
o
o
CO
^—^
-t-4
^—^
^•-^
aj
too
^*-^
4—4
-t—»
0^
aj
t>0
03
Ct-H
aj
aj
>
• 1-H
4—>
CO
03
X
CJ
o
Z
03
X
o
o
Z
O
PQ
in
VH
s*-^
s—^
aj
-t-4
CO
4-4
03
0^
aj
-t-j
o
U
CJ
3
fe
X
o
aj
^
CO
s»^
-t-4
4-4
g
-4—>
CJ
CJ
CH
aj
aj
o
_>
z
OJ
OJ
OJ
da j
C4X
aj
aj
cJX
«4-H
aj
aj
aj
aj
_>
.>
_>
|-4-4
-4-4
4—'
'w
'co
CO
O
O
^
O
0
OH
OH
OH
OH
OH
OH
OH
OH
^
CO
s ^
^
^
CO
3^
(N
3;^
a
a
a
a
03
03
VH
VH
o
o
b
U
^a j
C4-I
aj
too
C
^-1
¥ ¥
o
s-*^
4-4
CJ
o
OH
en
"^H.*^
aj
>
• 1-H
4-4
CO
¥ ¥
3
a
a
a
a
X
03
cd
o
03
o
VH
o
od
X
aj
aj
Z
aj
>
4—»
o
s^-^
-1-4
aj
-1-4
o
U
•^
¥
a
a
03
VH
o
o
VH
>
s—^
b
g
Jd
• —H
4—>
• —H
•4—"
>
_>
CO
CO
'co
VH
03
a
a
3
cd
CJ
o
VH
O
>
aj
aj
N—^
C
0
• 1-H
4—>
03
'c
03
too
4-J
0
¥
a
a
ed
0
VH
VH
0
<
>-,
B
o
raxo
fl
<
in
CO
CO
ro
''cr
CO
VH
aj
4-4
-1-4
Cd
0^
aj
-1-4
o
CJ
03
CJ
'B
X
OJ
aj
fe
VH
CO
Jd
3
-2
03
o
>
03
303
XCJ
O
M
303
X
a
a
a
a
VH
cd
VH
Cd
>
o
>o
o
1-H
fs
f^
Tf
O
o
o
o
o
4—4
aj
c
o
-1-4
U
fe
a
a
VH
cd
o
VO
<
03
3
z
Xed
CJ
O
fe
a
a
VH
03
a
>
VH
0
03
• ^H
•4—>
a
a
03
303
too
VH
ed
0
VH
0
13
aj
=3
*-t-4
o
U
OH
00
ON
aj
I
et
©
H
Z
ON
ON
o
o
322
«r)
o
VO
o
r-o
00
ON
0
-.
o
o
^H
^H
03
VH
aj
03
0^
;ir
^^
ctj
<
g
ro
4—"
OJ
<
<
g g
c^r
c,
<
ri
r^i
r^,
-4—»
•4—>
4—>
•4.J
4—*
OJ
CJ
CJ
g
•4—>
CJ
0
ro
aj
?a
M-H
^CH-H
^< 4 - l
^C4-H
c^
bb
X
aj
aj
aj
aj
aj
aj
aj
aj
tX)
OJ
13
aj
_>
X
-t—'
• 1-H
CO
c
o
VH
_>
>
4—'
4—>
CO
CO
S
O
O
OH
OH
ro
ro
CO
4-*
4-4
4—>
o
o
o
aj
aj
aj
aj
aj
_>
_>
-1-4
4—"
O
OH
«4-l
13
OJ
X
s
^
QX
aj
OJ
>
•^
'JO
0
OJ
r^,
g
^—'
"cj
,'^
^
''-^M
5x
aj
aj
OJ
aj
_>
>
•
4—1
CO
OH
ro
CN^
^
c
0
OH
aj
<
0-
>
-t-4
Cd
too
aj
z
E
©
cd
P^
o
<
z
aj
s
'^
VH
aj
03
OH
g
CJ
<
g
<u
X
-4—4
<
4-4
ctx
13
aj
PQ
ro
VH
03
a
a
ed
VH
o
^-^
-4-4
c
aj
-4-4
c
o
U
>
<
g
y:1
^
¥
JJ
X
aj
aj
VH
o
_>
CX
•t—>
0
cia
«
aj
aj
_>
CO
CO
• 1—1
4-4
CO
CO
0
CO
O
O
O
O
aj
0
OH
OH
OH
OH
(N
^^
^•-^
C3
^>—^
_o
g
4-J
c
aj
c
o
U
4—>
-4-4
03
N
303
^.-^
-t-4
c
aj
-4-J
c
o
U
tiO
VH
:^
VH
03
a
a
03
VH
o
4—>
C4-I
in
^—^
Avoidanc
aj
•t-4
H
-4—>
<N
^*-^
in
^^—^
CO
CJ
0
303
4—>
X
CJ
•S
aj
03
c3
3too
VH
4—>
03
N
• ^H
03
•t—>
CJ
<
g
aj
CJ
303
X
CJ
aj
too
VH
03
a
a
cd
VH
0
4—'
0
>
'"
s
:^
ro
aj
aj
13
OJ
X
CO
0
0-
OH
<N
CO
o
•t—>
^^^
^—^
4—>
^-^
4-J
c
aj
c
0
4.J
-4—>
U
c
aj
c
0
U
VH
o
0
tion (2;
CA
VH
tion (2;
<
©
X
et
03
N
03
N
<i:
s^
E
<
X
et
VH
aj
H
4—'
Cd
0<
VH
-4-4
03
a
a
ed
VH
o
aj
-t-4
c
o
U
03
N
VO
<
g
• ^H
-1-4
-1-4
aj
aj
-t-4
C5
O
U
03
tiO
-t-j
C
o
U
m
Jd
3cd
303
X
CJ
O
>
VH
o
13
aj
ry(3:
c
©
tion (2;
©
CO
CJ
X
o
aj
3too 3
ed
VH
o
03
in
CO
CJ
303
X
OJ
fe
4—>
a
a
aj
aj
4-4
•t—4
VH
U
U
TT
<N
in
fS
03
aj
0
r<
<s
m
<s
too
•t—4
C
0
c
0
VH
0
3
C
•4—>
c
o
u
1—1
OH
aj
X
cd
H
<s
^
et
©
1-H
f^
TT
m
VO
f-
1-H
H Z
323
00
ON
o
<s
1-H
r^
H—^
r^i
'—^
"—'
too
^--^
OJ
'•4—r
OJ
aj
CO
CO
OJ
OJ
OJ
aj
aj
Cd
rCJ
>
O
• —
4—>
CO
Z
'y^
O
C
O-
r^i
r<-,
^-^
-4—'
^—^
ffec
flee
c
OH
aj
^
OJ
too
<
r<-,
<
g
^-^
g
fee
g
r^,
cd
%
—
OJ
CJ
OJ
X
itiv
oi
OJ
o'
lee
"cd
^-.
<
itiv
r^i
lee
V_0J
itiv
CO
Teet (
PC
z
CO
o.
C
O-
^
S
ro
<
^
rti
•t—>
g
^._^
O
o
U
>.
-4-J
03
oi
g
^—^
.._«
aj
OJ
aj
aj
_>
_>
>
4—>
-4—>
'-t-4
• 1-H
t/)
CO
CO
aj
aj
O
o
CX
o
X
o
o
Z
aj
O
OH
OH
VH
aj
aj
>
4—»
VH
O
OH
OJ
>
3
3>
Cd
too
OJ
<
aj
03
Oi
^^
^-4—>
—^
C
^
^—^
VH
03
g
a
-4—4
c
o
U
^^
^—^
• ^ j
c
•t-4
ram
VH
o
U
a
•^
VH
03
a
acd
VH
o
z
r-
Tfi
03
PQ
<
too
;^
too
OJ
O
OH
. eff
et
H
- ^ H
r^i
ffee
aj
<
ang(
VH
oblem
<
X
ffee
ono
E
z
in
^—^
CO
o
3
' ^
^—^
-4—>
c
aj
,
^.^^
-*—*
c
aj
^
VH
03
•4-4
•t—>
X
o
o
U
o
U
a
a
cd
o
in
^-
_
^
•4—'
•t—'
4—>
•t—>
c
o
U
o
U
03
aj
<
g
<
>^,
E
©
X
03
^
<
g
CO
•t-4
aj
-t-4
c
o
U
>
13
b
X
>
CO
OJ
• 1-H
C
03
X
Mec
b
VH
VO
oca ula
Rater
'=1-
V.
Cd
UIU
H
CO
Grar
<
oca ula
E
©
V-
VO
<
g
cd
V.
a
OJ
ZJ
r-
c
o
U
—
0^
JJ
3
03
C<i
e«
H H
•
©
VO
fS
r-~
<s
00
<N
ON
*s
o
m
Z
324
^
r*^
«M
f^
m
m
Tf
in
f*^
m