AMEP Assessment Task Bank
Professional Development Kit
How to Develop an Assessment Task for Speaking Skills
Notes to accompany the PowerPoint for the AMEP Assessment Task Bank Professional
Development Kit. Developed by Marian Hargreaves for NEAS, 2013.
Slide 1: Front page
Slide 2: Aims of this workshop
•
•
To consider issues specific to assessing speaking skills
To develop skills in designing and developing assessment tasks for speaking skills
for the CSWE
• To validate speaking assessment texts and tasks.
There are also a number of activities suggested, and this presentation is really just a lead-up
to some practical and relevant hands-on experience.
Slide 3: Specific issues related to assessing speaking:
In addition to the general principles that apply to all assessment, there are a number of
specific issues that need to be considered when assessing speaking skills. These relate to:
1. Designing the task
2. Administering the task: issues relating to the interlocutor
3. Assessing the learner: issues relating to rating/reliability
Slide 4:
1. Designing the task:
Requirements of the Learning Outcome need to be explicit
Speaking is in real time. Unlike reading and writing there is little opportunity for drafts and
revisions. Therefore, although the requirements of the CSWE curriculum for achievement of
the learning outcomes need to be understood for all modules and macro-skills, in a speaking
task it is especially important that the requirements are explicit.
The criteria are set out very clearly for the teacher in the CSWE curriculum and it is
important that the learners are also aware of these criteria. This means that the criteria
themselves need to be taught and practised.
Slide 5:
For example, CSWE 2 Module C LO 2 Participate in a spoken transaction for
information/goods, or services. This learning outcome has 10 criteria. All the criteria have to
be achieved for the learner to pass this LO but unless the learner is aware of all the
requirements s/he will probably not pass the assessment.
© NEAS Ltd 2014
1
Slide 6:
Opportunity to meet requirements of the criteria
It is important that in the task the learner has the opportunity to meet these requirements.
For example, if the criteria include giving a name and address, the learner must be asked for
their name and address. If the criteria require the learner to demonstrate checking skills,
then the task must have this opportunity built into it. Several tasks require that the learner
make notes, therefore the learner must be aware of this for the assessment, and be given
the tools to make notes, eg pencil and paper/a form to complete/a diary entry to make.
If possible, there should also be a real communication gap so the interlocutor does not know
exactly what the student is going to say. For example, if the learner is describing a picture,
the interlocutor should not know which picture the learner is about to describe. Similarly, if
the interlocutor does not know the order of the cues given to the student, then there is a
great test of the learner’s intelligibility. (see notes for Slide 12)
Context and motivation
Given the settlement focus of the AMEP it is important to give a relevant context to the task.
Learners need to be able to navigate a spoken transaction or conversation that is meaningful
and relevant to them such as making a doctor’s appointment or booking travel for
themselves and family. This means that they have more opportunity and motivation to
practice and use the skills learnt in class.
Collaborative
Speech acts often involve other people but are not necessarily collaborative in as much as
although speakers always need listeners, they do not always need someone to talk back in
an exchange. For example, C1 F2 Give a spoken description; C1 J2 Tell a short recount; C2
D2 Give information in a simple presentation; C3 E2 Deliver a short spoken presentation.
This can make providing a relevant context more challenging but these are skills that are
potentially very important in the workplace. Learners may need to recount the events of an
accident or give a short presentation on some aspect of their work. Learners can and should
be encouraged to provide appropriate contexts.
Tasks need to comparable
For achievement in the CSWE a learner needs to provide two pieces of evidence. Tasks
should therefore be designed to be comparable with a similar number of requirements in a
similar style.
For example, CSWE 2 C2 Participate in a spoken transaction for information/goods, or
services. ‘Camera’ and ‘Computer Printer’ both clearly give the context, instruct the learner
with a clear task, and provide the opportunity for the learner to give personal details and for
the interlocutor to note those details.
Suggestion: give the details of these two tasks as a handout (provided below at the
end of the document) so that workshop participants can see clearly what is meant.
Slide 7: The support given for a task should also be similar. For example, a scenario without
prompts is a much more difficult task than a scenario with prompts.
© NEAS Ltd 2014
2
In addition to the formal assessment tasks, observation and informal assessment are also
valid assessment strategies that can be used in the CSWE curriculum. If you observe a
student undertaking an authentic task that addresses all the criteria for a spoken Learning
Outcome you can use this performance for assessment as long as you can provide the
evidence that it addresses all criteria. An observation checklist, such as those provided on
the Assessment Task Bank, is one way of achieving this evidence. So, if you take your
students to a market, for example, and observe them in an authentic transaction as opposed
to a role play, this could be a valid assessment performance.
Slide 8: Planning and pre-teaching
Assessment needs to be an integral part of the teaching program. With a speaking task it is
important that the teaching is specifically designed to culminate in an assessment that is as
stress-free as possible. This is important especially for reliability and fairness. The
assessment should come as no surprise either in topic and style. Practice as often as
possible is therefore particularly important. This can be a good opportunity for peer
assessment and formative evaluation with criterion-based feedback so that the learner can
have every chance to produce the language that is expected for achieving the learning
outcome. Both role-plays and authentic interactions are good opportunities for integral and
informal assessment.
Familiarity with the experience is very important. Speaking is a performance with the added
stress that this entails. With reading and writing, assessment is usually done in a class, and
the performance is very anonymous – until the results are given, no one need know how
badly you perform. However, speaking is very different and even a competent speaker may
freeze in a formal assessment situation. The assessment event should therefore be as
informal as possible using natural functional tasks as appropriate.
If the learner is to be recorded then this should be practised too so that the learner is
completely comfortable with the process. It is also a useful teaching tool for instant feedback
both to the individual and to the whole class, eg giving a description.
Slide 9:
2. Administering the task: issues relating to the interlocutor
This is a good point to look at the role of the teacher in assessment in the AMEP. When a
learner is being assessed for a speaking learning outcome the teacher can be both
interlocutor and assessor/rater; just the rater, while someone else performs the role of
interlocutor; just the interlocutor with another person rating the performance, or even both
interlocutor and rater, but making the assessment later by listening to the recording of the
performance.
Although this presentation recognises the distinction between assessor/rater and
interlocutor, for a teacher in the AMEP it is necessary to be able to perform either or both
roles.
Interlocutor
© NEAS Ltd 2014
3
Being an interlocutor for a speaking assessment is not at all the same as having a chat with
someone. The interlocutor must be very familiar with the requirements of the task, give the
learner every opportunity to produce the required language, and let the learner speak! The
interlocutor should actually speak as little as possible and give time for the learner to
generate questions and responses.
When more than one student is being assessed, the interlocutor needs to be as consistent
as possible across all the students.
Who is an Interlocutor?
The interlocutor is responsible for conducting the spoken assessment. S/he elicits the
spoken performance of the learner by asking them questions or setting up speaking tasks.
The interlocutor may be a native-speaker or non-native speaker of English. They must have
a good command of English and be able to elicit language at an appropriate level from the
learner.
The interlocutor is not necessarily the assessor. However, in the AMEP the interlocutor is
often also the assessor. For this reason, spoken assessments should, if possible, be
recorded.
What qualities and skills does an interlocutor need?


a thorough understanding of the language and linguistic forms being tested at each
level
attentive to the administrative demands of the role

familiarity with the task being used

skilful in elicitation

patient and encouraging

keep to the time limit of the test (if applicable)

know how to operate recording equipment effectively.
As already noted, the interlocutor for a learner in the AMEP is often their teacher or another
teacher from the teaching institution but they do not have to be. The curriculum merely
stipulates that the “transaction is with a known interlocutor who in fluent in English, eg
volunteer, other teacher, class teacher.” However, when inviting someone to be an
interlocutor you need to be aware that a teacher of migrants with low-level English language
skills will be more attuned to their speech, than, say a native speaker who is happy to
volunteer for the role. This will affect the intelligibility of the learner and therefore affect the
assessment. If the interlocutor is not familiar with the speech of the learners, including the
accents they bring from their L1, then the interlocutor should not be responsible for making
the assessment decision.
What are the responsibilities of an interlocutor? (Note – volunteers may not need to be
involved with the more administrative aspects of the assessment eg room set-up)
Interlocutors should

ensure that the room is set up for assessment
© NEAS Ltd 2014
4

check the equipment used for the recording

be completely familiar with the criteria of the Learning Outcome of the assessment


familiarise themselves with the assessment before the first learner starts the
examination
manage the interaction at the test and facilitate the learner's performance

adjust their level of English as is appropriate for the level of the assessment

keep to the prescribed time for all parts of the assessment (if applicable)

ensure that all learners are treated equally

ensure that all the administration is carried out accurately

take regular breaks between a number of assessments

complete the necessary administration in connection with the assessment task.
General advice to interlocutors
Suggestion: this section (general advice) could be
made into a handout – adapted/simplified as
required.

Be as friendly as you can.
Smile at the learner when he or she comes into the room and indicate where you
want them to sit. Be friendly and encouraging throughout the assessment, try to put
the learner at ease.

Respect the learner’s right to privacy. Be aware of not asking questions around very
personal or culturally sensitive topics.

Give clear and effective instructions.

Give the learner every chance to speak. Don’t rush the learner. If learners fall silent,
allow them a few seconds to organise their thoughts. Only prompt if you are sure they
need help. Some pausing and hesitation is a natural feature of spoken English at all
levels.

Be an active listener and show interest.

Behave consistently with each learner.

Don't correct mistakes and only supply language when you have allowed the student
sufficient time to search for it.
Be careful about being too helpful eg rephrasing and paraphrasing what the student
has said as could affect the rater’s assessment of the performance as they tend to
tune into the ‘extra’ information given by the interlocutor.


Avoid asking closed questions which only require a ‘yes / no / one word’ reply
Ask open ‘wh’ questions (what, why, when, where, which, how).


Don't dominate the interaction! Respond and contribute where appropriate, but give
the learner every opportunity to perform to his / her best ability.
Don't pitch your language level too high or too low.

Don't ask questions that test general knowledge.

Tasks are often scripted in order to achieve this consistency. If not scripted, then
tasks must be clearly structured, both in the rubrics and in the scenario.
Rater/Assessor
© NEAS Ltd 2014
5
What qualifications and experience does a successful assessor need?
In addition to the skills of an interlocutor, the assessor or rater should have


appropriate English language teaching qualifications proven professional
competence
teaching experience at an appropriate level
Slide 10: Recording
It is often difficult to be an effective interlocutor and an efficient rater simultaneously. It is
therefore highly advisable to record a performance so that it can be reviewed later especially
if achievement is in some doubt.
Recording may be required for record-keeping purposes. It is also useful for training
sessions, moderation meetings and feedback.
Recording the spoken assessment
The equipment
Tape recorder or digital?
It really doesn’t matter what technology you use, so long as
 you know how to work it
 the student is familiar with being recorded and the process does not add to the stress
of being assessed.
Labelling
The recording must be clearly labelled, ideally both in written form, and in verbal form at the
start of each recording.
Each recording should record
 the date
 the Certificate level, Module and Learning Outcome
 the name of the student.
Notes on administration
 Do not stop the recording of a learner once you have started.
If there are unexpected interruptions such as a fire bell ringing or someone
accidentally entering the Test Room, leave the machine running and allow for the
required extra time.
 If there is a time limit to the assessment, have a silent clock or stopwatch on
the desk in a discreet position and, in a gentle and friendly way, keep the learners to
the specific timings allowed.
 Tell the learner to speak up if the voice level is low or quiet. Do not turn off the
recording machine when you say this. Such interruptions will not affect the
assessment of the learner, turning the machine on and off is more likely to distract
the student and lead to a confused recording.
Slide 11: 3. Assessing the learner: issues relating to rating/reliability
Spoken performance is rated rather than marked. The assessment criteria of the CSWE
attempts to meets the requirements of validity and reliability and standardise assessment by
using an assessment grid.
© NEAS Ltd 2014
6
The Assessment Grid
Using the assessment grid – refer to PD Kit Speaking Handout - CSWE II G2 Assessment
Grid which is on the website.
This grid lists all the criteria for the learning outcome, and all of the criteria must be achieved
(A) for the learner to be successful. This is a useful reminder of all the skills that the learner
must demonstrate.
But expressions such as ‘effectively’ and ‘appropriately’ are not clarified. These assume
considerable professional knowledge in teaching English language, and experience in the
CSWE curriculum.
There is also no opportunity, using the Assessment Grid, to comment on adverse testing
conditions (eg roadworks outside, or interruptions during testing).
If the criteria specify that a learner demonstrate a specific skill (eg where name and contact
details are required), but the student is not asked for them and the interlocutor and/or learner
task prompts does not indicate that they should give them, the assessment grid gives no
opportunity for any kind of note to be made.
Rating
This puts significant pressure on examiners (raters) to be able to maintain consistency both
between the assessments of individual learners (intra-rater reliability) and with other raters
(inter-rater reliability). An element of subjectivity is unavoidable.
The best ways to achieve reliability are
1. Pre-testing moderation,
2. Awareness of subjectivity.
Pre-testing moderation generally assumes that there will be more than one assessor/rater.
Benchmark and borderline samples of speaking performances can be assessed and
differences of opinion can be discussed and resolved. This also resolves different
interpretations of the criteria (eg what is ‘effective’?)
If you are the only assessor, you should ensure that you are completely familiar with the
criteria and the certificate level. Listen to recordings of a few benchmark performances to
help attune yourself to what can be expected from the learner.
Ways to avoid subjectivity include




Not assessing your own students – you know them too well and are used to their
manner of speech
Built-in communication gaps in the task
Recording all the learners so that a second opinion can be given for borderline
performances
Taking regular breaks so that you don’t get tired.
Slide 12: Intelligibility
Intelligibility is a criterion in all levels of the CSWE curriculum, including Preliminary.
© NEAS Ltd 2014
7
This is a complex concept, including pronunciation, word stress, intonation, pauses, context
and even body language. “Two people can judge the intelligibility of the same speaker
considerably differently.” (Yates and Springall 2008)
However, a simple moderation exercise can raise the awareness of the issue and make the
evaluation of student performances more reliable.
Refer to Assessing Intelligibility by Lynda Yates and Jacky Springall (on website).
Materials in this resource address issues such as:
•
•
•
•
•
•
The difference recognisable/ intelligible?
Is there a scale of intelligibility?
How do you account for listener factors?
– Familiarity
– Sympathy
– Interlocutor knowledge of the task/cues
How do you identify sound clarity?
Does pausing and intonation only matter at CSWE 3?
How do you help less experienced teachers cope with assessing pronunciation?
Slide 13: Feedback
All assessment, including summative assessment for achievement should provide an
opportunity for feedback as an important part of the learning process. Feedback should be
given following every occasion of assessment.
Feedback can be given either in writing or verbally, but should be confidential at all times,
except where a whole class activity is undertaken, in which case anonymous feedback
should be given to the whole class. However, as it is very important to keep all documents in
a summative assessment task set secure for future assessment purposes, answer keys
should not be given out, nor learners allowed to keep their corrected response sheets.
Slide 14
Workshop Activities
Activity 1 Brainstorming
This is a ‘first stages’ activity for a group interested in developing an assessment task or
activity.
Activity 2 Build a bank of materials
This could follow on from Activity 1 and is especially useful for Preliminary and CSWE 1
assessment.
Activity 3 Designing a task for C2 G2 Participate in a simple interview
This is a specific task development activity for a designated Learning Outcome.
Activity 4 Design a task for C 3 C2 Negotiate a complex spoken exchange
© NEAS Ltd 2014
8
This is another specific task development activity, with some suggestions regarding
context/topic. It is also a good opportunity to raise awareness of the importance of the
learner/interlocutor communication gap.
© NEAS Ltd 2014
9
Workshop Activities
Activity 1
Brainstorming
This is a ‘first stages’ activity for a group to identify and consider what CSWE level they want
to develop assessment tasks/activities for; which learning outcomes they want to address;
what style of assessment would be suitable; what scenarios would be relevant for the
learners.
This activity is particularly useful for teachers less experienced in developing assessment
tasks, or who are new to the curriculum.
This activity is also a useful way of learning the requirements of the CSWE curriculum.

Identify the CSWE level that you want to assess

Identify the learning outcome/s for the level that you want to design tasks for

Think about the interactive or collaborative nature and requirements of the LO

Check existing tasks for what has already been done, on the ATB, in sample AMES
tasks, in-house tasks

Brainstorm scenarios. Consider purpose and context of the task

Check the curriculum

Consider whether another angle on a topic already with a task could be suitable,
eg for CSWE 3 C2 Negotiate a complex spoken exchange

Look at other LOs in the module eg CSWE 1 Module E listening and speaking skills
for short informal spoken exchanges

Consider the style of existing tasks as a model for designing comparable tasks

Think about alternative forms of assessment eg observations, role play, peer and
self-assessment, and where these might be appropriate.
© NEAS Ltd 2014
10
Activity 2
Build a bank of materials
This activity could follow on from Activity 1 and is especially useful for Preliminary and
CSWE 1 assessment.
Learning Outcomes that require a good bank of materials include:






Preliminary Modules B & C: alphabet and numbers
Preliminary Module D: visual symbols and signs
Preliminary Module E: numbers, time, money
Preliminary Module H: simple words, simple sentences
Certificate I Module F: descriptions
Certificate I Modules L and M: time, date, currency, measurement, graphs, maps,
shapes etc
1. Evaluate existing material and which LOs they apply to
2. Brainstorm potentially useful materials. These could include pictures, realia, and
models of benchmark performance
3. Consider relevant topics eg road safety (RTA materials), Emergency services,
4. Look at the settlement topics
5. Check the DIAC website for recent materials and publications
http://www.immi.gov.au/living-in-australia/help-with-english/amep/teaching-english
6. Delegate members of the group to source new materials
7. Reconvene to build an outline of lessons and practice building up to assessment
using these materials.
Remember: validity, reliability and practicality!
© NEAS Ltd 2014
11
Activity 3
Designing a task for CSWE 2 G2 Participate in a simple interview
AMES tasks include an interview at TAFE, a tenancy interview with a real estate agent, and
a generic job interview. The curriculum suggests tasks for a job interview, or a real estate
agent interview.
Using the format of the ATB/AMES tasks as a model, design a similar task for a job as:





a cleaner in a motel;
a farm worker picking blueberries/cherries/apples
a horticultural worker in a nursery
a shelf stacker in a supermarket
a job that your learners might want to apply for in your locality
Remember to check the curriculum (provide as a handout)
Instructions and background information must be clear and unambiguous.



Get feedback from other teaching colleagues
Pilot with learners
Modify task according to feedback.
© NEAS Ltd 2014
12
Activity 4
Design a task for CSWE III C2 Negotiate a complex spoken exchange
This LO already has tasks for extending the due date for an assignment, changing the time
of a class, and returning a faulty headset (ATB), plus transferring to an evening class at
college, taking a day off work, reporting a faulty ATM (which has failed to deliver your
money) to the bank (AMES). The curriculum also suggests tasks for changing a work shift,
changing an appointment, dealing with a problem such as rental accommodation etc.
A scenario might be:
On moving into your rental accommodation the learner finds that the
apartment is not clean although the condition report states that the
flat has been found in a clean state.
The learner has to phone the real estate agent, complain about the condition of the flat and
arrange for the flat to be cleaned. This could include another inspection by the agent/details
about the lack of cleanliness eg mould in the bathroom, dirty oven, stained
curtains/engagement of a professional cleaner/rent concession for cleaning
activities/suitable times for any appointment etc.



Check the criteria for expected length of performance, complexity required etc.
Check other tasks for the learning outcome to ensure comparability
Develop the task. Ensure that the task set contains
o Interlocutor guidelines (as per existing tasks)
o Assessment procedure (as per existing tasks)
o Interlocutor role card
o Leaner role card.
Instructions and background information must be clear and unambiguous.



Get feedback from other teaching colleagues
Pilot with learners
Modify task according to feedback.
Develop another task related to rental accommodation, which could be used as a further
piece of evidence for achievement of this learning outcome within the same topic.
Eg phone the real estate agent to complain about a faulty toilet cistern. The learner could
give details of the fault/ask for an inspection/ask for a plumber/ask for a new toilet/arrange a
time for an appointment etc.
Consider integrating the topic with another learning outcome eg CSWE III C1 Demonstrate
understanding of a complex spoken exchange. As part of the development of a task for
CSWE III C2, a script could be developed and recorded of a spoken exchange between a
landlord and a rentor of an apartment. A listening task could then be developed on this topic,
and contribute towards the completion of the module.
In addition, the NSW Fair Trading ‘New tenant checklist’ might form the basis for a task for
CSWE III H1 Demonstrate understanding of a complex written information sheet.
Handout
© NEAS Ltd 2014
13
CSWE II C2 Participate in a spoken transaction for information, goods or services.
Camera
© NEAS Ltd 2014
14
Computer Printer
© NEAS Ltd 2014
15