CHAPTER 7: ASSESSING SPEAKING
-
BASIC TYPES OF SPEAKING
Imitative (parrot back);
Intensive;
Responsive;
Interactive;
Extensive.
1. Imitative: (parrot back) testing the ability to imitate a word, phrase, and
sentence Pronunciation is being tested. Examples: Word, phrase, sentence
repetition.
2. Intensive: The purpose is producing short stretches of oral language. It is
designed to demonstrate competence in a narrow band of grammatical, phrasal,
lexical, phonological relationships (stress / rhythm / intonation) Examples of
intensive assessment tasks include directed response task, reading aloud,
sentence completion, limited picture cued tasks and sentence level translation.
3. Responsive: (Briefly interacting with the interlocutor) Responsive assessment
tasks include interaction and test comprehension but at the somewhat limited
level of very short conversations,standars greetings and small talk, simple
requests and comments, and the like.
4. Interactive: The difference between responsive and interactive speaking is the
length and complexity of the interaction, which sometimes includes multiple
exchanges and/or multiple participants. Interaction in the form of
either transactional language or interpersonal exchanges is tested. (In the
three dialogues above A and B are transactional, and C is interpersonal.
Examples: Interviews, role plays, games, discussions.
5. Extensive (monologue): Extensive oral production tasks include speeches, oral
presentations, and story-telling, during which the opportunity for oral
interaction from listeners is either highly limited (perhaps to nonverbal
responses) or ruled out together.
MICRO AND MACRO SKILLS OF SPEAKING
Micro skills of speaking
Pronounce the distinctive sounds of a language clearly enough so that people can
distinguish them. This includes making tonal distinctions.
Use stress and rhythmic patterns, and intonation patterns of the language clearly
enough so that people can understand what is said.
Use the correct forms of words. This may mean, for example, changes in the tense,
case, or gender.
Put words together in correct word order.
Use vocabulary appropriately.
Use the register or language variety that is appropriate to the situation and the
relationship to the conversation partner.
Make clear to the listener the main sentence constituents, such as subject, verb,
object, by whatever means the language uses.
Make the main ideas stand out from supporting ideas or information.
Make the discourse hang together so that people can follow what you are saying.
Macro Skills of Speaking
Appropriately accomplish communicative functions according to situations,
participants, and goals.
Use appropriate styles, registers, implicature, redundancies, pragmatic conventions,
conversion rules, floor keeping and yielding, interrupting, and other sociolinguistic
features in face-to-face conversations.
Convey links and connections between events and communicate such relations as
focal and peripheral ideas, events and feeling, new information and given
information, generalisation and examplification.
Convey facial features, kinesics, body language, and other nonverbal cues along with
verbal language.
Develop and use a battery of speaking strategies, such as emphasizing key words,
rephrasing, providing a context for interpreting the meaning of words, appealing for
help, and accurately assessing how well your interlocutor is understanding you.
DESIGNING ASSESSMENT TASK: IMITATIVE SPEAKING
Imitative speaking refers to ability of the students in speaking performance of simply
parrot back or imitates a word, phrase, or possibly a sentence. The appropriate
assessment tasks for this level are:
- Word repetition task
- The scoring procedure is:
+ 2 for acceptable pronunciations.
+ 1 for comprehensible, partially correct pronunciation
+ 0 for silence, seriously incorrect pronunciation
DESIGNING ASSESSMENT TASK: INTENSIVE SPEAKING
Test-takers are prompted to produce short stretches of discourse (no
more than a sentence) through which they demonstrate linguistic ability at
a specified level of language. Many tasks are “cued” in that they lead the
test-taker into a narrow band of possibilities. Some of the techniques
include
 Directed respond task
 Read- aloud task
 Sentence/dialogue completion tasks and oral questionnaires
 Pictured-cued tasks
 Translation (of limited streches of discourse)
DIRECTED RESPOND TASK
These are mechanical, non-communicative tasks; but they do require
minimal processing of meaning in order to produce the correct
grammatical output.
READ-ALOUD TASKS
These tasks include reading beyond the sentence level up to a paragraph or
two.
Advantages: predictable output, practicality, reliability in scoring
Disadvantages: somewhat inauthentic in that we seldom do this in real life,
also this skill calls for certain specialized oral abilities which may not be
reliable indicators of test-taker’s pragmatic ability to communicate orally in
face-to-face contexts
Prator’s (1972) Manual of American English Pronunciation’s diagnostic passage:testtaker read aloud into a recorder, scoring based on a number of phonological factors
(vowels, diphthongs, consonants, consonant clusters, stress, and intonation) with a
two-page diagnostic checklist on which all errors and questionable items were noted
and a four point scale for pronunciation and for fluency.
ASSESSING INTENSIVE SPEAKING:
SENTENCE/DIALOGUE COMPLETION TASKS
Include reading a dialogue in which one speaker’s lines have been omitted. Other
examples are form filling (Underhill 1987) or oral questionnaires.While individual
variations in responses are accepted, this technique taps into a lerner’s ability to
discern expectancies in a conversation and to produce sociolinguistically correct
language
It could be contended that performance on these items is responsive rather than
intensive but notice there is a degree of control which predisposes the test-taker to
respond with certain expected forms. In any event, according to Brown this
argument undersacores the fine lines of distinction between the five categories for
assessing spoken language Brown (2004).
Advantages: moderate control of output, the written format allows a little bit more
time for the test-taker to anticipate the answer and removes the potential ambiguity
created by aural misunderstanding.
Disadvantages: the contrived, inauthentic nature of this task and the fact that it relies
on literacy and an ability to transfer easily from written to spoken English.
Example: Dialogue completion task
Example: Dialogue completion task
Assessing intensive speaking:
Picture-cued tasks
Designed to elicit a word or phrase. Pictures may be very simple, somewhat
more elaborate and “busy”, or composed of a series that tells a story or
incident.
Other cues: Maps to give instructions, directions, and specify locations.
Other techniques: pairing two test-takers supplied with identical sets of
numbered pictures. One test-taker is cued to describe one of the pictures
in as few words as possible for the other test-taker to identify.
Advantages: help to unlock the almost ubiquitous link between listening and
speaking performance and remove the potential ambiguity created by aural
misunderstanding.
Disadvantages: the inauthentic nature of this task and the fact that it relies
on literacy and an ability to transfer easily from written instructions to
spoken English.
Also, although this technique is quite versatile but it can be heavily
dependent on very clear written instructions.
Scoring: may be problematic depending on the expected performance
Picture based tasks are very popular to elicit oral language performance and
can be used not only for intensive production but also for extensive output.
When scoring multiple factors recordings of the test-takers productions are
very useful to the grader.
 Types of language that can be elicited using pictures:
 minimal pair
 comparatives
 verb tenses
 nouns, negative responses, numbers, and location
 giving directions and instructions
 elaborate responses and descriptions
Example: Picture-cued elicitation task
minimal pairs
minimal pairs
comparatives
Intuitive: do not rely on written instructions
I
ntuitive: do not r These pictures need
clear written instructions as they could
be misleading and confusing without
them.
ely on written instructions
c
ASSESSING INTENSIVE SPEAKING:
TRANSLATION (OF LIMITED STRETCHES OF DISCOURSE)
According to Brown, translation methods are certainly passé in today’s
communicative classroom; but he concedes that in countries (such as
Venezuela) where English is still not a prevailing language translation plays a
meaningful communicative device for the English learner.
This technique involves test-takers being given a native language
word,phrase,or sentence and are asked to translate to the English
equivalent.
Advantages: control of the output which of course means that scoring is
more easily accomplished.
ASSESSING RESPONSIVE SPEAKING:
Differs from intensive tasks in the increased creativity given to the testtaker and from interactive tasks by the somewhat limited length of
utterances.
Involves brief interactions with an interlocutor
Some of the techniques commonly used include:
 Question and Answer
 Giving instructions and directions
 Paraphrasing
ASSESSING RESPONSIVE SPEAKING:
QUESTIONS AND ANSWERS
These tasks can consist of simple and complex questions from an interviewer or they
can make up a portion of a whole battery of questions and prompts in an oral
interview. There are two types of questions
 Display questions: this is the first question and it intensive in its purpose (as
we have seen previously, this questions are designed to elicit a predetermined
correct response).
 Referential questions: through these questions the test-taker is given
opportunity to produce meaningful language in response.
In designing referential questions it is important to keep in mind why the question is
being asked; is it to elicit a string of language output or is it to gain a sense of the
test-taker’s discourse competence?
Oral interaction with a test administrator often involves the latter asking all
questions. An alternative of this concept is to elicit questions from the test taker.
One technique involves more than one test-taker with an interviewer. With two
students in an interview context, both test-takers can ask questions of each other.
This technique might meet practicality requirements but it might be troublesome to
score.
Example: Question and answer task
Questions eliciting open-ended responses
Elicitation of questions from the test-taker
ASSESSING RESPONSIVE SPEAKING:
GIVING INSTRUCTIONS AND DIRECTIONS
This technique is simple, the administrator poses the problem and the test-taker
responds.
Task should require the test taker to produce at least 5 or 6 sentences
Topics need to be familiar (not beyond the content schemata of the test-taker), so that
an impromptu delivery is attainable, this avoids having to supply the problem in
advance which in turn guarantees the test-taker does not parrot back a memorized set
of sentences.
Advantages: Using this type of stimulus provides an opportunity for the test-taker to
engage in a relatively extended stretch of discourse, to be very clear and specific, and
to use appropriate discourse markers and connectors.
Scoring: based primarily on comprehensibility and secondarily on other specified
grammatical or discourse categories.
Example: Giving instructions and directions task
These tasks can be designed to be simple or complex, potentially placing it in
the category of extensive speaking. Objectives must be clearly set if the purpose is to
elicit a short and simple response directives must be clear so as not to take the testtaker down a path of complexity for which she or he is not prepared.
Assessing responsive speaking:
Paraphrasing
This tasks require the test-taker to read or hear a limited number of sentences
and produce a paraphrased version of the discourse.
It is important to pinpoint the objective of the task clearly. In these tasks the
integration of listening and speaking is probably more at stake than simple oral
production alone.
Advantages: elicit short stretches of output and perhaps tap into test-takers’
ability to practice the conversational art of conciseness by reducing the output/input
ratio.
Some of the contexts that may be assessed include: Describing, comparing and
contrasting, narrating, summarizing, giving an opinion, supporting an opinion,
hypothesizing, defining, functioning “interactively”.
Assessing interactive speech:
Include long stretches of interactive discourse. The difference between these
types of oral production assessment and responsive speech is the length and
complexity of the expected output. Can take two forms:
 Transactional language: to exchange specific information
 Interpersonal exchanges: social exchanges and relationships
Some of the techniques commonly used include:
Interviews
Role plays
Discussions
Games
ASSESSING INTERACTIVE SPEECH:
INTERVIEW
This technique involves a test administrator and a test-taker sitting down
in a direct face-to-face exchange and proceeding through a protocol of questions
and directives. Interview can vary in length, depending on their purpose:
Placement interview: designed to get a quick spoken sample from a
student in order to verify placement into a course, may need only five minutes if the interviewer is trained to evaluate the output accurately.
Comprehensive interviews: designed to cover predetermined oral
production contexts and may require the better part of an hour.
A variation is to place two test-takers during one interview, the
advantages of this technique are the opportunity for student-student interaction
which increases authenticity and the practicality of scheduling twice as many
candidates. The disadvantages are equalizing the output between two testtakers, discerning the interaction effect in case of unequal comprehension and
production ability, and scoring two people simultaneously.
Scoring: based on a set of parameters which may include accuracy in
pronunciation, grammar, vocabulary usage, fluency, sociolinguistic/pragmatic
appropriateness, task accomplishment, and even comprehension. Scoring can
be facilitated by recording the interview.
This technique involves a test administrator and a test-taker sitting down
in a direct face-to-face exchange and proceeding through a protocol of questions
and directives. Interview can vary in length, depending on their purpose:
Placement interview: designed to get a quick spoken sample from a
student in order to verify placement into a course, may need only five minutes if the interviewer is trained to evaluate the output accurately.
Comprehensive interviews: designed to cover predetermined oral
production contexts and may require the better part of an hour.
A variation is to place two test-takers during one interview, the
advantages of this technique are the opportunity for student-student interaction
which increases authenticity and the practicality of scheduling twice as many
candidates. The disadvantages are equalizing the output between two testtakers, discerning the interaction effect in case of unequal comprehension and
production ability, and scoring two people simultaneously.
Scoring: based on a set of parameters which may include accuracy in
pronunciation, grammar, vocabulary usage, fluency, sociolinguistic/pragmatic
appropriateness, task accomplishment, and even comprehension. Scoring can
be facilitated by recording the interview.
Disadvantages: open-ended and involves a significant level of interaction
where the interviewer is forced to make judgments that are susceptible to
unreliability. Accuracy in scoring can be improved with careful attention to the
linguistic criteria being assessed as well as through experience and training of
the administrators to develop a sound judgment.
The success of an oral interview will depend on:
 clearly specifying administrative procedures of the assessment
(practicality)
 focusing the questions and probes on the purpose of the assessment
(validity)
 appropriately eliciting an optimal amount and quality of oral production
from the test-taker(biased for best performance)
 creating a consistent, workable scoring system (realibility)
According to Brown, every effectiv
interview contains a number o
mandatory stages.
Two decades ago Michael Cana
(1984) proposed a framework whic
has withstood the test of time.
Canale suggested that test-takers w
perform at their best if they are le
through four stages:
Example: Interviews
Assessing interactive speech:
Role play
This is a popular pedagogical activity in communicative language teaching
classes.
Advantages: it can be controlled or ‘’guided’’ by the interviewer while freeing
students to use discourse that might otherwise be difficult to elicit allowing testtakers to go beyond simple intensive and responsive levels to a level of creativity and
complexity that approaches real-world pragmatics.
Scoring: presents the usual complications as any task that elicits somewhat
unpredictable responses from test-takers.
Assessing interactive speech:
Discussions and conversations
Difficult to specify and even more difficult to score.
Advantages: as informal techniques to assess learners, they offer a level of
authenticity and spontaneity that other assessment techniques may not provide.
Discussion is a integrative task, so it is advisable to give some cognizance to
comprehension performance in evaluating learners.
Scoring: checklists should be carefully designed to suit the objectives of the
observed discussion
Discussions may be specially appropriate tasks through which to elicit and
observe such abilities as:
Assessing interactive speech:
Games
Among informal assessment devices are a variety of games that involve
language production. Some examples include:
 ’’Tinkertoy’’ game
 Crossword puzzles
 Information gap grids
 City maps
Advantages: as informal techniques to assess learners, they offer a level of
authenticity and spontaneity that other assessment techniques may not provide.
Scoring: checklists should be carefully designed to suit the objectives of the
observed discussion
Example: Games
Assessing extensive speech:
These tasks involve complex, relatively lengthy stretches of discourse. They are
frequently variations on monologues, usually with verbal interaction from listeners or
an interlocutor being either highly limited or ruled out all together. Some commonly
used techniques include
Some of the most commonly used techniques include:
Speeches and oral presentations
Pictured cued story-telling
Retelling a story or news event
Translation (of extended prose)
Assessing extensive speech:
Oral Presentations
These tasks consist of having the test-taker present a report, a paper, a marketing
plan, a sales idea, a design of new product, or a method.
Scoring: checklist and grid are common means of scoring these tasks. Scoring is the
key assessment challege for oral presentations so the rules for effective assessment
must be invoked
 Specify the criterion clearly
 Set appropriate tasks
 Carefully elicit optimal output
 Establish practical,reliable scoring procedures
Assessing extensive speech:
Picture-cued story-telling
These tasks are similar to those we reviewed for assessing intensive production. The
object is to elicit oral production through visual cues. Some of the stimuli used
include:
Pictures
Photographs
Diagrams
Charts
Series of pictures for longer descriptions
Scoring: criteria need to be clear about what is being assessed. For example it is
insufficient to specify the objective as aiming to elicit narrative discourse. This must
be further clarified by deciding whether the assessment is evaluating oral vocabulary,
time relatives, sentence connectors, past tense of irregular verbs, etc.
Example: Picture-cued elicitation task
for extensive production
Possible questions:
Example: Picture-cued elicitation task
1. Who is eating?
for extensive production
2. Who is drinking?
3. Who is talking?
This task elicits more open-ended
4. What is she doing?
performace whereby test-takers
In applying questions it is im
have to elaborate with their own
know the purpose of each
opinion, describe preferences, and
The purpose of the first thr
accomplish a persuasive function.
is to cue the test-taker towa
These tasks must have a clearly
Assessing extensive speech:
Retelling a story or news event
In these tasks test-takers hear or read a story or news event that they are asked to
retell.
The difference from the paraphrasing is longer stretches of discourse and a different
genre.
Scoring: the most significant challenge as with all extensive production assessments,
therefore it should be designed to meet a clear set of criteria.
Some commonly used rubrics include communicating sequences and relationships of
events, stress and emphasis patterns, ’’expression’’ in the case of a dramatic story,
fluency, and interaction with the hearer.
In these tasks test-takers hear or read a story or news event that they are asked to
retell.
The difference from the paraphrasing is longer stretches of discourse and a different
genre.
Scoring: the most significant challenge as with all extensive production assessments,
therefore it should be designed to meet a clear set of criteria.
Some commonly used rubrics include communicating sequences and relationships of
events, stress and emphasis patterns, ’’expression’’ in the case of a dramatic story,
fluency, and interaction with the hearer.
Assessing extensive speech:
Translation (of extended prose)
Longer texts are presented for the test-taker to read in the native language and then
translate into English. Some of examples of texts include:
 Dialogues
 Directions for assembly of a product
 A synopsis of a story or play or movie
 Directions on how to find something on a map
Advantages: is in the control of the content, vocabulary, and to some extent, the
grammatical and discourse features.
Disadvantage: as we know, translation of text is a highly specialized skill for which
some individuals obtain advanced degrees.
Scoring: criteria should therefore take into account not only the purpose in eliciting a
translation but the possibility of errors that are unrelated to oral production ability.
Longer texts are presented for the test-taker to read in the native language and then
translate into English. Some of examples of texts include:
 Dialogues
 Directions for assembly of a product
 A synopsis of a story or play or movie
 Directions on how to find something on a map
Advantages: is in the control of the content, vocabulary, and to some extent, the
grammatical and discourse features.
Disadvantage: as we know, translation of text is a highly specialized skill for which
some individuals obtain advanced degrees.
Scoring: criteria should therefore take into account not only the purpose in eliciting a
translation but the possibility of errors that are unrelated to oral production ability.
Longer texts are presented for the test-taker to read in the native language and then
translate into English. Some of examples of texts include:
 Dialogues
 Directions for assembly of a product
 A synopsis of a story or play or movie
 Directions on how to find something on a map
Advantages: is in the control of the content, vocabulary, and to some extent, the
grammatical and discourse features.
Disadvantage: as we know, translation of text is a highly specialized skill for which
some individuals obtain advanced degrees.
Scoring: criteria should therefore take into account not only the purpose in eliciting a
translation but the possibility of errors that are unrelated to oral production ability.
Final comments
Oral Proficiency scoring categories (Brown 2001)
Phonepass ® (imitative and intensive) vs.
TSE (responsive and interactive) vs.
OPI (oral interview)
References
Brown, H.D. Language Assessment: Principles and Classroom Practices. (2004).
Longman
It refers to the use of pictures or sequence of pictures. This method is effective to
stimulate speaking respond of the test takers. Administrator provides some picture
with some clues, and then the test takers point to the picture to give the answer or
teacher point to the picture and then test takers mention the answer orally.
There are some kinds of picture cued task such as:
Ø Picture cued elicitation of minimal pairs (test administrator points to each picture
in series by using question ‘what is this?’)
Ø Picture cued elicitation of comparatives (use a comparative form to compare some
objects in the picture)
Ø Picture cued elicitation of future tense (using question form of future tense)
Ø Picture cued elicitation of nouns, negative response, number, and location (the
administrator point to the picture and then test takers answer by mentioning what the
picture is)
Ø Picture cued elicitation of response and description (test takers describe what does
the picture about.
DESIGNING ASSESSMENT TASK: RESPONSIVE SPEAKING
Ø Question and answer. It refers to short interview.
Ø Giving instruction and direction. In this case, the administrator poses the problem
and then the test takers respond it. The scoring is primarily based on the
comprehensibility, and secondarily on another grammatical category.
Ø Paraphrasing. Test takers read or hear the text or story and the paraphrase it by
using their own words.
DESIGNING ASSESSMENT TASK: INTENSIVE SPEAKING
Ø Interview. It refers to the oral conversation in which the administrator and the test
takers sit down face to face to do question and answer. It is design to get the quick
spoken sample from the student in order to verify the placement in to a course.
Ø Role play. This technique is very popular in communicative language teaching
classes. It is like mini drama in which the test takers act out to perform some
characters in particular story.
Ø Discussion and conversation
Ø Games
DESIGNING ASSESSMENT TASK: EXTENSIVE SPEAKING
Ø Oral presentation. In this case, the test takers are ordered to arrange a particular
paper or assignment and then present the materials in front of the class.
Ø Picture cued story telling. Test takers are given the sequences of the picture story
and then create a story based on the pictures.
Ø Retelling a story, news event. In this case the test takers hear the story and then
retell the story orally
Ø Translation of extended prose
Diposting oleh Unknown di 05.11
Kirimkan Ini lewat EmailBlogThis!Berbagi ke TwitterBerbagi ke FacebookBagikan
ke Pinterest
Tidak ada komentar:
Posting Komentar
Posting Lebih BaruPosting LamaBeranda
Langganan: Posting Komentar (Atom)
Arsip Blog
 ▼ 2013 (5)
o ▼ Juni (4)
 review advance novel



o
WORD FORMING PROCCESS COINAGE, BLENDING, ACRONYM
ASSESSING SPEAKING
ASSESSING LISTENING
► Mei (1)
IMITATIVE: It is simply testing the ability to imitate a word, phrase, and
sentence. Pronunciation is being tested. It focuses on pronunciation. Not
concerned much about comprehension or expression of meaning
E.g. Repeat after me: [bit, beat]
I bought a boat yesterday
many believe that nonmeaningful imitation of sounds is fruitless. Such opinions have
faded in recent years as we discovered that an overemphasis on fluency can
sometimes lead to the decline of accuracy in speech. So, we have emphasized
pronunciation, especially of suprasegmentals, in an attempt to help learners be more
comprehensible.
An occasional phonologically focused repetition task is warranted as long
as repetition tasks are not allowed to occupy a dominant role in an overall oral
production assessment and as long as you artfully avoid negative washback.