ASSESSING WRITING (2)
Lecture 9
Teaching Writing in EFL/ESL
Joy Robbins
TODAY’S SESSION

The portfolio system of assessment



What is portfolio assessment?
What are the pros and cons of portfolio assessment?
How do I set up portfolio assessment?
We’ll also look at some recent studies of students’
attitudes toward portfolio assessment
2
WHAT IS THE PORTFOLIO SYSTEM OF
ASSESSMENT?



A portfolio is a collection of writing, normally a selection of
the student’s writing that they have done over their course.
Hyland (2003) defines portfolios as ‘multiple writing samples,
written over time, and purposefully selected from various
genres to best represent a student’s abilities, progress, and
most successful texts’ (p.233).
The pieces of writing which go into the portfolio are selected
from the whole of the student’s work
The portfolio is evaluated by the teacher (see Yancey 1992)
What do you think are the advantages and
disadvantages of using a portfolio system to
assess your students’ writing, rather than
using traditional exams?
3
ADVANTAGES:
PORTFOLIOS AND PROCESS WRITING


(1)
Fits in much better with process writing pedagogy
than exams or writing tests. With exams, students
have little chance to draft and redraft. Using
portfolio assessment, teachers can ask students to
write multiple drafts, and the final drafts can go
into the portfolio.
Portfolios therefore encourage good (process)
writing habits: like revision. Sommers (1991) puts
it well: ‘The portfolio itself tends to encourage
students to revise because it suggests that writing
occurs over time, not in a single sitting, just as the
portfolio itself grows over time and cannot be
created in a single sitting’ (p.153-4)
4
ADVANTAGES:
PORTFOLIOS AND PROCESS WRITING



(2)
Another feature that portfolios share with process
writing is they both promote dialogue
The portfolio system also fits in with the process
approach’s emphasis on peer collaboration and
reflection (Burnham 1986).
Nunes (2004) argues that portfolios should
‘facilitate on-going interaction between teacher and
students. If the portfolio is to be conceived as an
instrument of dialogue, it cannot be written in one
session and handed in at the end of the academic
year. On the contrary, it has to be continually in
the making and document work in progress’ (p.328)
5
ADVANTAGES:
PORTFOLIOS AND PROCESS WRITING


(3)
Nunes (2004) also argues that portfolios
encourage students to reflect on their writing
In her article, Nunes describes how she
encouraged her learners to include reflections
on their writing practices in their portfolios,
as well as assignments
6
ADVANTAGES:
PORTFOLIOS AND CONTINUITY


Portfolios allow students to write a series of texts
that build on each other. Exam writing is
disconnected (Black et al 1994; Yancey 1992)
Portfolios allow writers to view the progress they
are making and their development as writers
(Bailey 1998; Burnham 1986)
7
ADVANTAGES:
PORTFOLIOS AND EVALUATION

Portfolio advocates also often believe that the
way writing is evaluated is more in line with
process pedagogy, because students’ work is
commented on throughout the drafting stages
8
ADVANTAGES:
PORTFOLIOS AND EVALUATION


(2)
‘Portfolio evaluation establishes a writing
environment rather than a grading environment
in the classroom’ (Burnham 1986:139).
More constructive teacher feedback is possible
using a portfolio system. Because the teacher
does not have to assign grades early in the
writing process, comments can be more
constructive than the comments used in grading
writing exams (Burnham 1986; Elbow 1993;
Myers 1996; Peterson 1995).
9
ADVANTAGES:
PORTFOLIOS AND EVALUATION


(3)
‘By delaying the appraisal of written
products…to the end of an academic term, a
portfolio approach promotes revision,
encouraging students to assume responsibility for
their learning by giving them control over how
they manage their time’ (Ferris & Hedgcock
2005: 320-1)
And other researchers claim that this delayed
assessment takes the anxiety out of writing
for some students who fear evaluation (e.g.
Johnston 1983)
10
ADVANTAGES: PORTFOLIOS AND THE
TEACHER—STUDENT RELATIONSHIP

Summarizing the work of a number of scholars (e.g.
Condon & Hamp-Lyons 1991; Smit et al 1991;
Sommers 1991), Baker (1993) says that ‘The teacher
seems to be cast in a different light—that of a
mentor or coach—instead of the idiosyncratic
authority figure who assigns grades. Teachers are
less likely to put grades on papers written in a
portfolio classroom than in a standard processcentered classroom, and they instead focus on
comments which students are more likely to read
and follow because students are given additional
opportunities for revision’ (p.156)
11
ADVANTAGES: PORTFOLIOS AND THE
TEACHER—STUDENT RELATIONSHIP (2)

So the idea is that teachers become readers of
students’ writing, making constructive
comments, rather than assessors…
This may lead to a better, more respectful classroom
atmosphere…
12
ADVANTAGES:
PORTFOLIOS AND STUDENT RESPONSIBILITY


A portfolio ‘places a large measure of control over
success into the learner’s hands’ (Hamp-Lyons &
Condon 2000:35)
Ferris & Hedgcock (2005) make the point that
the delay in evaluation until the end of term
means that the learners’ destiny is, to some
extent, in their own hands, because they have the
time to work on their drafts as much or as often
as they wish
13
PORTFOLIOS: DISADVANTAGES
WORKLOAD


A common argument put forward against the
portfolio system (see Smit et al 1991; Sommers
1991) is that it may result in more work for
teachers
For instance, when Baker (1993) recalls how she
asked some writing teachers at her university to
begin assessing students by portfolio, she notes
that some teachers ‘expressed concern that their
paperload might increase dramatically’,
‘[p]erhaps envisioning students writing five or six
drafts of each essay in an attempt to improve
their work’ (p.162)
14
PORTFOLIOS: DISADVANTAGES
GRADE INFLATION


Baker (1993) also mentions that some of the
teachers feared the portfolio system would result
in ‘grade inflation’, where higher grades become
more common because everyone has the chance
to revise their work a number of times (p.162)
So some teachers may feel a portfolio system
encourages dumbing-down, and environment
where everybody passes, and there are lots of A
grades
15
PORTFOLIOS: DISADVANTAGES
TEACHER UNEASE AT THEIR LACK OF
CONTROL?

And some researchers also claim that some
teachers won’t feel comfortable with the
more student-centred classroom that will
result from the portfolio system
16
PORTFOLIOS: DISADVANTAGES (2)

Ensuring reliability is a massive problem: would
two teachers give the same grades for the same
piece of work in a portfolio?
Reminder: Reliability
 ‘reliability refers to the consistency with which a
sample of student writing is assigned the same
rank or score after multiple ratings by trained
evaluators’. (Ferris & Hedgcock 1998: 230)

In other words, if we’re marking an essay out of 20,
the test will be far more reliable if two markers
both award an essay the same grade (or more or
less the same grade), say 16 or 17. However, if one
marker awards 10 and the other awards 15, the
test isn’t reliable.
17
DISADVANTAGES (3)



Still on the reliability issue…How can teachers
reliably arrive at a single mark which fairly
assesses the whole of the student’s portfolio? The
portfolio is likely to feature writing of a number of
different genres…
How can two students’ portfolios which feature
totally different genres of writing be fairly
compared? (Grabe & Kaplan 1996)
There are always potential problems with
plagiarism and peer collaboration: ‘How will
the portfolio raters know that the students
actually wrote all the pieces in the portfolio, and
when is editing and revising assistance from other
too extensive to represent the student’s own
writing abilities’ (Grabe & Kaplan 1996: 417)?
18
PORTFOLIO ASSESSMENT AND
TEACHING CONDITIONS

Lam & Lee (2010) explain why portfolios aren’t
popular in Hong Kong:
‘the exam-oriented culture in Hong Kong has made
it difficult for innovative pedagogical ideas, such as
process pedagogy to flourish…. Multiple drafting is
considered a luxury because teachers are hardpressed to cover the syllabus to help students
prepare for…exams. Second, most practising
teachers have not received training in the
implementation of…portfolio programmes. They
tend to think that asking students to document all
their drafts in a folder and grading it summatively
amounts to [portfolio assessment].’
19
This reminds us that a great deal of care and
preparation will be needed if it is decided to
switch over to the portfolio system…
20
DECIDING TO CHANGE TO PORTFOLIO
ASSESSMENT
Hyland (2003: 236-7) offers a number of useful questions the
teacher needs to be able to answer if they’re going to move to a
portfolio system of assessment. Here are a few of them:
1.
Who will choose which pieces of writing go into the
portfolio? Teachers only? Students only? Teacher and
student together?
2.
Should the entries receive a preliminary initial grade or
the portfolio only be graded as a whole?
3.
What part will students’ reflections and self-assessments
play in the assessment?
4.
How will consistent scoring and feedback be achieved—
what rater training is needed?
5.
How many people will grade the portfolio and how will
scoring disagreements be resolved?
6.
What mechanisms should be set up for evaluating the
program and making changes to it?
If you were moving over to a portfolio system, what
would your answers be to these questions?
21
PORTFOLIOS & THE IMPORTANCE OF
LEARNER TRAINING



As Hyland (2003) goes on to argue, if you do
decide to move over to portfolio assessment,
learner training is vital. Students need to be
trained how to:
Select items to put into their portfolios
Write reflective comments on their choices. The
teacher will probably ask for these comments to
be included in the portfolio
How could you ensure students knew
how to write reflective comments?
22
RESEARCH ON THE USE OF PORTFOLIOS
What does it tell us about portfolios’
effectiveness, and about attitudes towards
the portfolio system of assessment?
23
STUDENTS’ ATTITUDES TOWARDS
PORTFOLIO ASSESSMENT

Baker (1993) notes that students ‘praised the use
of portfolios and often mentioned their changed
attitude toward writing’ which was a result of the
introduction of portfolios (p.167)
24
STUDENTS’ COMMENTS


One student said portfolios ‘give students a feeling of
accomplishment’; another student said he ‘felt more
confident in what [he] wrote’; another said the
portfolio classroom environment ‘provided a more
relaxed atmosphere for me to write in’ (p.167)
There was also evidence that students appreciated
the fact that grades were now delayed: one student
commented
‘I think the portfolio method helps students out.
Instead of receiving a bad grade on a paper a student
has a chance to revise their paper’ (p.167)
25
STUDENTS’ COMMENTS

When Baker (1993) asked the students whether
teachers should continue using the portfolio
system, over 90% said yes.
However, it should be noted that the students
in Baker’s study were undergraduate native
speakers…
As Hamp-Lyons & Condon (2000) point out,
what’s noticeable is that there have been few
studies of portfolios in EFL/ESL contexts…
26
PORTFOLIOS IN HONG KONG
But Lam & Lee (2010) is an EFL study
investigating how students and teachers
reacted once portfolios had been introduced
to their classroom. They report positive
attitudes…
27
THE OPPORTUNITY TO CONFERENCE

‘I like the teacher consulting section most. It is
because it motivates students to seek teachers’
feedback about their performances. Students and
teachers can have a chance to exchange their
ideas’. (Lam & Lee 2010: 59)
28
THE BENEFITS OF A PROCESS-ORIENTED,
SUPPORTIVE WRITING APPROACH (1)

‘…in the past, we wrote the composition for the
sake of passing the exam and we had only an
hour to write it. Now, we have more time and
support such as multiple drafting, peer feedback,
conferences and so forth…’ (Lam & Lee 2010: 59)
29
THE BENEFITS OF A PROCESS-ORIENTED,
SUPPORTIVE WRITING APPROACH (2)

‘In secondary school, as teachers adopted [a] oneshot approach to writing, I was only expected to
produce one draft and there wasn’t much time for
deeper thinking’. (Lam & Lee 2010: 60)
30
ANOTHER STUDY OF PORTFOLIOS IN
EFL/ESL CONTEXTS


Song & August (2002) is another of the few
EFL/ESL studies available, and is quantitative
As a result of comparing non-native university
writers’ pass rates on a timed exam and a portfolio,
with the portfolio producing a higher pass rate,
Song & August (2002) argue that timed writing can
unfairly penalize non-native writers
31
TIMED WRITING VS. PORTFOLIOS

‘These exams particularly handicap ESL
students because they not only test them on
unfamiliar genres and tasks, but also require
them to meet standards of excellence in
grammatical and mechanical accuracy they
cannot reach on a first draft in 50 minutes…’
(Song & August 2002: 61)
32
Another study:
Hirvela & Sweetland (2005)
33
SETTING OF THE STUDY
 Two
undergraduate writing programmes in
an American university which required
students to produce a portfolio
 The
portfolios were only worth 5% of the
students’ overall grade for one course, and
10% for the other, in an effort to get students
to focus less on grades, and more on
lecturers’ formative assessment (and
students’ self-assessment) of the writing
process
34
THE REFLECTIVE LETTER

For both courses, ‘the students were required to
write an introductory letter in which they were to
discuss the writing contained in the portfolios by
reflecting on what the samples revealed about
their development as writers’ (Hirvela &
Sweetland 2005: 196)
The idea was to find out about the students’
experiences of, and attitudes towards, the
two different ways portfolios were used on
the courses…
35
METHODOLOGY
Data about students’ experiences and
attitudes was collected via
 interviews
 analysis of their reflective letter
 (students’ pieces of writing in their portfolios was
also examined)
Two students took part in the study…
36
STUDENT 1: SHIM
Korean, doing a business degree
 Had never been asked to keep a portfolio of
writing before

37
STUDENT 2: MOTO
Japanese, doing an Aeronautical Engineering
degree
 Had moved with family to the US three years
before he started university, so had some
experience of doing portfolio writing at his
American high school

38
SHIM’S REFLECTIVE LETTER (1)
 ‘The
students were instructed to (a)
identify, and provide a rationale for, the
pieces of writing they had chosen to
include in their portfolio, and (b) reflect on
the writing growth they had experienced
during the course’. (p.201)
However, it was clear that Shim didn’t
really engage with the task in this
way…
39
SHIM’S REFLECTIVE LETTER (2)

Shim explained the reflective letter as follows:
“I think the…letter is kind of, as you know, when
you make thesis in graduate school, before we
submit thesis, we have to write introductory
letter, right? I think this letter is like that,
abstract for my portfolio, and thanks for teaching
me, you are a nice teacher—I think the
introductory letter contains this kind of
information”. (p.201)
So Shim’s letter mainly summarized the
writing samples which he put in his
portfolio and thanked his teacher…
40
THE PURPOSE OF A PORTFOLIO:
FORMATIVE OR SUMMATIVE? (1)
 In
the interviews, Shim
‘maintained his belief…that a portfolio is
more appropriately used as the basis for
teachers to assess students than for
students to reflect upon their learning as
writers. In other words, he conceptualized
the portfolio…as summative in nature
despite its stated formative intentions’.
(p.202)
Note: Students were provided with
lengthy written explanations about
the formative nature of the portfolio!
41
THE PURPOSE OF A PORTFOLIO:
FORMATIVE OR SUMMATIVE? (2)

Moto, like Shim, saw the portfolio as summative:
it ‘was just another piece of schoolwork that
teachers used to “keep students busy”.’ (p.204)
42
THE PURPOSE OF A PORTFOLIO:
FORMATIVE OR SUMMATIVE? (3)
 Moto
didn’t see any point in selecting work,
and said he didn’t benefit from doing so
 He
also felt the reflective letter was a waste
of time and ‘boring’ (p.205)
‘Perhaps not surprisingly, then, he claimed
no sense of ownership for the submitted
portfolio…and did not feel proud of it’ (p.204)
However, it turned out that Moto kept
his own portfolio…
43
MOTO’S OWN PORTFOLIO

This consisted of all the pieces of writing Moto
had done for a number of years, and
‘he reported “enjoying” reading them once in
while now that he had become a more mature
and experienced…writer…he reported that
someday…he thought that his future job could
require him to do some of the types of writing
required [on the university] course, in which case
the work in his own portfolio could serve as
models of these tasks. […] [Moto] felt that he
would need an extended period of time as well as
a comprehensive collection of his work to assess
his growth as a writer.’ (pp.204-5)
44
THE PURPOSE OF A PORTFOLIO:
FORMATIVE OR SUMMATIVE? (4)
 ‘Because
[Moto] believed that future
reflection was more valuable than more
immediate reflection, he saw no reason
(beyond getting a higher grade) to spend time
trying to make the portfolio look good […] He
felt that students need more than the few
months a writing course typically covers in
order to fully internalize what they’re
learning about writing, and so they aren’t in
a position to reflect meaningfully on their
growth while compiling a portfolio’. (p.206)
45
REFLECTIONS ON WRITING
 ‘…development
of writing ability is a longterm process, whereas classroom
portfolios operate in the short-term realm
of a writing course. This may well be why
the introductory letters weren’t richer in
terms of their reflective comments. The
participants were being asked to reflect on
their growth as writers before they felt
they were ready to do so…’ (p.208)
46
A HANDS-OFF APPROACH TO
PORTFOLIOS
Shim and Moto’s lack of enthusiasm may also
be connected to the way the portfolio system
was implemented
 ‘…very little class time was devoted to discussing
portfolios in general and the portfolio
assignments …. The students were pretty much
left to their own devices in terms of interpreting
the portfolio intentions and requirements […] It
appears…that the…instructors failed…to
nurture an ongoing portfolio culture that would
enable students to better understand what the
portfolio pedagogies were meant to achieve…’
(p.209)
47
A FORMATIVE CULTURE…IN THEORY

Although the portfolios were meant to be
formative rather than summative, Hirvela &
Sweetland (2005) note that the classroom culture
wasn’t really formative…
So it’s not really surprising that Shim and
Moto saw the portfolios as summative…
48
WRITING ASSESSMENT: PREFERRED
METHODS




Based on last week’s lecture and this week’s session,
would you prefer to assess your students’ writing
ability by means of exams or the portfolio system?
Why?
Do you think the portfolio system may be culturally
more acceptable to western students than to students
from, say, south-east Asia? Why (not)?
If you are a language teacher, would your students be
happy with being assessed by means of the portfolio
system? Why (not)?
If you were learning a foreign language, would you
prefer your writing to be assessed by exam or by
portfolio? Why?
49
REFERENCES
Bailey KM (1998) Learning about Language Assessment: Dilemmas, Decisions,
and Directions. Boston: Heinle & Heinle.
Baker NW (1993) The effect of portfolio-based instruction on composition
students’ final examination scores, course grades, and attitudes toward
writing. Research in the Teaching of English 27(2): 155-174.
Black L et al (1994) New Directions in Portfolio Assessment. Portsmouth:
Heinemann.
Burnham C (1986) Portfolio evaluation: room to breathe and grow. In C.
Bridges (ed.), Training the New Teacher of College Composition. Urbana:
National Council of Teachers of English, pp.125-139.
Condon W & Hamp-Lyons L (1991) Introducing a portfolio-based writing
assessment: progress through problems. In P.Belanoff & M.Dickson (eds.),
Portfolios: Process and Product. Portsmouth: Boynton Cook, pp.231-247.
Elbow P (1993) Ranking, evaluating, and liking: sorting out three forms of
judgment. College English 55: 187-206.
Ferris D & Hedgcock JS (1998) Teaching ESL Composition: Purpose, Process,
and Practice. Mahwah: Lawrence Erlbaum.
Ferris D & Hedgcock JS (2005) Teaching ESL Composition: Purpose, Process,
and Practice (2nd edition). Mahwah: Lawrence Erlbaum.
Grabe W & Kaplan RB (1996) Theory and Practice of Writing. London:
Longman.
Hamp-Lyons L & Condon W (2000) Assessing the Portfolio: Principles for
Practice, Theory and Research. Cresskill: Hampton Press.
Hirvela A & Sweetland YL (2005) Two case studies of L2 writers’ experiences
across learning-directed portfolio contexts. Assessing Writing 10: 192-213.
50
REFERENCES (2)
Hyland K (2003) Second Language Writing. Cambridge: Cambridge University
Press.
Johnston B (1983) Assessing English. Sydney: St Clair Press.
Lam R & Lee I (2010) Balancing the dual functions of portfolio assessment.
ELT Journal 64: 54-64.
Myers M (1996) Sailing ships: a framework for portfolios in formative and
summative systems. In R. Calfee & P. Perfumo (eds.), Writing Portfolios in
the Classroom: Policy and Practice, Promise and Peril. Hillsdale: Lawrence
Erlbaum Associates, pp.149-178.
Nunes A (2004) Portfolios in the EFL classroom: disclosing an informed
practice. ELT Journal 58(4): 327-335.
Peterson R (1995) The Writing Teacher’s Companion: Planning, Teaching, and
Evaluating in the Composition Classroom. Boston: Houghton Mifflin.
Smit D et al (1991) Implementing a portfolio system at Kansas State
University. In P.Belanoff & M.Dickson (eds.), Portfolios: Process and Product.
Portsmouth: Boynton Cook, pp.46-56.
Sommers N (1991) Bringing practice in line with theory: using portfolio
grading in the composition classroom. In P.Belanoff & M.Dickson (eds.),
Portfolios: Process and Product. Portsmouth: Boynton Cook, pp.153-164.
Song B & August B (2002) Using portfolios to assess the writing of ESL
students: a powerful alternative? Journal of Second Language Writing 11:
49-72.
Yancey KB (1992) Portfolios in the Writing Classroom. Urbana: National
Council of Teachers of English.
51
THIS WEEK’S READING
Chapter 8 of:
Ferris D & Hedgcock JS (1998 or 2005) Teaching ESL
Composition: Purpose, Process, and Practice. Mahwah:
Lawrence Erlbaum.
If you’re interested in portfolios, also look at what Ken
Hyland (2003) says about portfolios in his book, Second
Language Writing,
and:
Hamp-Lyons L & Condon W (2000) Assessing the
Portfolio: Principles for Practice, Theory and Research.
Cresskill: Hampton Press.
Hirvela A & Sweetland YL (2005) Two case studies of
L2 writers’ experiences across learning-directed
portfolio contexts. Assessing Writing 10: 192-213.
52
NEXT WEEK
In the final session, we’re going to have a look at
using technology to teach writing
So next week’s class will be in computer lab G
53