The Effectiveness of Corrective
Feedback on Chinese EFL Writers’
Grammatical Accuracy Improvement
Dongmei Cheng
Northern Arizona University
Motivation

No general conclusion on the efficacy of
grammatical corrective feedback (e.g.
Truscott, 1996, 2007; Ferris, 1997, 1999,
2004, 2006)
Motivation

No consensus on how explicit feedback
should be:
–
–
More explicit feedback: Providing metalinguistic
information helps (Bitchener, 2008; Bitchener et
al, 2005; Lalande, 1982)
Less explicit feedback: Learners only need
guidance in locating the errors (Chandler, 2003;
Lee, 1997; Robb, Ross & Shortreed, 1986)
Motivation

Few studies has truly adopted “a sustained,
systematic, replicable manner that would
allow for comparisons across similar or
different contexts and student populations”
(Ferris, 2004, p.55).
Motivation

There’s a need to adopt a controlled design,
which has a comparable selection of
–
–
–
corrective feedback types
targeted error types
Writing tasks
Research Questions
RQ: Does two types of indirect corrective
feedback help to reduce the amount of
targeted grammatical errors in students’
argumentative writings over time? If both
types of feedback help, which one is more
effective?
•
•
Feedback type 1: Highlighting the targeted errors
Feedback type 2: Highlighting the targeted errors
and providing abbreviated error codes
Research Questions

Time (Within two months)
–
–
–
Time 1: First Draft of Essay 1 (Week Three)
Time 2: Revised Draft of Essay 1 (Week Five)
Time 3: First Draft of Essay 2 (Week Eight)
Participants
•
•
Total participants N=95
(Female N=60; Male N=35)
Participants came from three College English
classes taught by the same EFL instructor in
a big public university in Northern China.
(original participant pool N=120)
Participants
•
•
•
All college freshmen from three major areas:
Administrative Management, Economics and
International Accounting.
Have no regular English writing practices and
multiple-draft writing experiences.
Are generally competent English readers but
relatively weak in speaking and writing.
Participants
Age
Mean
18.69
College
Entrance
Exam Score
125.19
Standard
Deviation
0.75
12.76
Years of
English
Learning
8.39
1.87
Design


Two experimental groups + A control group
All learners in the three classes (initial participants’
pool N=120) were randomly assigned to three
different types of feedback:
–
–
–
Highlighting the targeted errors (Highlight)
Highlighting the targeted errors with abbreviated codes
(Code)
No grammatical feedback (None)
Procedure





All participants submitted their first essays
electronically through the school online course
platform.
The EFL instructor downloaded the essays and sent
them to the researcher.
The researcher provided grammatical feedback using
the “highlighter” and “comment” function on Microsoft
Word.
The researcher sent back the grammatically
commented essays to the EFL instructor.
The EFL instructor added in brief content and
organization feedback for all participants (including the
control group)
Variables

Independent variables:
–
–

Feedback type (Highlight, Code vs. None)
Time (Essay 1; Revised Essay 1 vs. Essay 2)
Dependent variable:
Targeted error score (per essay)=
occurrence of targeted error(s)*100
total word
Argumentative Writing Tasks


Two essay topics from the IELTS Task Two
Writing prompt pool.
Selection criteria:
–
–
Participants are familiar with both topics.
The two prompts fit in the “agree vs. disagree”
type, which provides a claim and asks the
participants to what extent they agree or disagree
with the given claim.
Argumentative Writing Tasks
Essay 1:
More and more women go out to work. Is it the
government’s responsibility to subsidize
them and provide free staff and facilities to
care for their children? To what extent do you
agree or disagree to this idea?
Argumentative Writing Tasks
Essay 2:
The government is responsible for protecting a
nation’s cultural identity. Thus, some people
believe new buildings should be built in
traditional styles. To what extent do you
agree or disagree with this opinion?
Targeted Errors—Type I
•
Type I:
•
Noun endings (NE)
e.g. This phenomenons become more and more
common among office ladies.
More and more woman go out to work.
Targeted Errors—Type II

Type II: Verbs
–
Subject-Verb Agreement (SV)


–
The government have responsibility to subsidize them.
Culture exist in every one’s spirit.
Verb Tense (VT)


As the society developed, more people want their lives
to be more comfortable and convenient.
To protect or rebuilt old buildings of traditional styles is
more meaningful than build some ones.
Targeted Errors—Type II

Type II: Verbs
–
–
–
Subject-Verb Agreement (SV)
Verb Tense (VT)
Verb Form (VF)


If new buildings be built in traditional styles, people will
pay more cost and can’t adapt to live in these buildings.
In order to protect the nation’s culture, what we really
need to do is to protect the treasure we have already
own, like old buildings, old paintings and so on.
Targeted Errors—Type III

Type III: Sentence Structure
–
Sentence Fragment (SF)




Here^ my reasons.
So the government should subsidize the women.
Because now people have a lot of pressure of modern
life.
And finding job is easier.
Targeted Errors—Type III

Type III: Sentence Structure
–
–
Sentence Fragment (SF)
Run-on Sentences (RO): (including comma
errors)



There is a woman go to work.
Some problems appeared, who will care for their
children.
A series of traditions are left which have a profound
effect on us.
Targeted Errors—Type IV

Type IV: Word Form
–
Wrong word formations including nouns,
adjectives and adverbs (WF)



The construct which represent a nation’s history are of
vital importance.
Mothers always show their children with patient.
When they get home, they also need to work hard to
keep the room cleanly.
Reliability Check


Intra-rater reliability: 25% of the essays were
coded again by the researcher after two
months. Pearson Correlation r= .97
Inter-rater reliability: 25% of the essays were
coded by a second coder.
Pearson Correlation r=.91
Results—Descriptive Statistics
Raw Error
Score
Essay 1
Revised
Essay 1
Essay 2
Highlight
M=5.74
SD=2.51
M=2.74
SD=2.39
M=5.29
SD=3.33
Code
M=5.74
SD=2.90
M=1.19
SD=1.38
M=3.48
SD=2.32
None
M=7.23
SD=3.43
M=6.30
SD=3.23
M=7.30
SD=3.98
Raw Error Occurrences
Results—Descriptive Statistics
Normed
Error Score
Essay 1
Revised
Essay 1
Essay 2
Highlight
M=3.69
SD=1.88
M=1.57
SD=1.40
M=3.29
SD=2.79
Code
M=3.54
SD=1.62
M=0.70
SD=0.86
M=2.07
SD=1.46
None
M=3.03
SD=1.52
M=2.72
SD=1.29
M=3.05
SD=1.48
Normed Error Scores
4
3.5
3
2.5
Highlight
2
Code
None
1.5
1
0.5
0
Essay1
RevEssay1
Essay2
Results—Group One (Highlight)



Alpha was set as .05.
Test: Within-subject repeated measure
ANOVA
RQ1: Does one type of indirect feedback
(Highlight) help to reduce the error scores in
students’ essays over time?
Results—Group One (Highlight)



F(2)=14.97, p<.05
There is a significant time effect on the
highlighting group’s error scores.
Results from pairwise comparisons:
–
–
Essay1 vs RevEssay1: p<.05
Essay1 vs Essay2: p>.05
Results—Group Two (Code)
•
•
RQ2: Does another type of indirect
feedback (code) help to reduce the
error scores in students’ essays over
time?
Results from repeated measure
ANOVA:
F(2)=33.58 p<.05
Results—Group Two (Code)


There is a significant time effect on the error
scores of participants’ who received coded
feedback.
Pairwise comparison results:
–
–
Essay1 vs. RevEssay1: p<.05
Essay1 vs. Essay2: p<.05
Results—Control Group

•
Repeated ANOVA results:
F(2)=.788, p>.05
There is no significant time effect on control
group’s error scores.
Results—Between-subject


•
Test: Between-subject repeated measure
ANOVA
Results:
F(2)=4.40, p<.05
There is a significant difference in the error
scores over times among the three groups.
Results—Between-subject

Pairwise comparisons:
–
–
–



Highlight vs. Code: p<.05
Highlight vs. None: p>.05
Code vs. None: p<.05
There’s a significant difference on error scores over
time between the two experimental groups.
The “Highlight” group showed NO significant
difference on error scores over time compared to the
control group.
The “Code” group showed a significant difference on
error scores over time compared to the control
group.
Error Scores from the Three groups
Over time
Conclusion



Both types of indirect feedback helped to reduce the
targeted errors in students’ argumentative essays
significantly in the short term (Essay1RevEssay1)
The more explicit feedback (Code) helped to reduce
the targeted errors significantly from Essay 1 to
Essay 2 (in a longer term)
Highlighting the targeted errors and providing
abbreviated codes had a more significant effect
(compared to only highlighting the errors) on error
reduction in students’ essays over time.
Pedagogical Implications



EFL students could potentially benefit from
more explicit corrective feedback on their
argumentative writings.
Providing grammatical codes helps the
students to notice and understand the nature
of their grammatical errors in the essay.
A mutual understanding on the meanings of
the error codes needs to be established
beforehand. (e.g. an error chart)
Mean Error Scores for Noun Ending
Errors
0.7
0.6
0.5
Highlight
0.4
Code
0.3
None
0.2
0.1
0
Essay1
RevEssay1
Essay2
Mean Error Scores for Verb Errors
0.9
0.8
0.7
0.6
Highlight
0.5
Code
0.4
None
0.3
0.2
0.1
0
Essay1
RevEssay1
Essay2
Mean Error Scores for Sentence
Structure Errors
2.5
2
1.5
Highlight
Code
None
1
0.5
0
Essay1
RevEssay1
Essay2
Mean Error Scores for Word Form
Errors
1.6
1.4
1.2
1
Highlight
0.8
Code
None
0.6
0.4
0.2
0
Essay1
RevEssay1
Essay2
Results for Word Form Errors

Within-subject repeated measure ANOVA
–
–
–

Highlight: F(2)=.039, p>.05no sig time effect
Code: F(2)=4.34, p<.05sig time effect
Control: F(2)=18.54, p<.05sig time effect
Between-subject repeated measure ANOVA
–
F(2)=2.51, p>.05no sig different difference
among the effect of the three feedback types on
the word form error scores
Implications of the unexpected
findings on word form errors



Learners perhaps have difficulties correctly
interpret the feedback on word form errors.
Compared to the other three types of
targeted errors, word form errors might be
relatively easier to be ignored by the
learners, despite the provision of feedback.
More explicit classroom instruction needs to
be done to address the correct use of word
forms in context (vocabulary instruction).