ELI listening / speaking diagnostic activities The rating scales for the oral presentation and group discussion activities Developed by Yusuke Fujisawa Fall 2004 Aspects to be rated oral presentation Fluency Intelligibility Vocabulary Organization/ coherence Delivery group discussion Fluency Intelligibility Vocabulary Communicative skills/strategies Discussion content Scale level definitions For each category, the scale points are to be matched to the level described below. scale point 4.0 3.0 2.0 1.0 0.0 For teachers level description sufficient level acceptable level close to acceptable level insufficient level not measurable For students level description excellent very good ok need more improvement not measurable Sufficient level / excellent level The level which is considered sufficient in real academic classes Acceptable level / very good level The level which is considered acceptable in real academic classes Close to acceptable level / ok level The level which is considered not acceptable in real academic classes, but very close to that level Insufficient level / need more improvement level The level which is considered insufficient in real academic classes 1 Detailed descriptors for each aspect Oral Presentation Fluency Key features Speaking rate Length of utterance unit Filled/unfilled pauses Natural/unnatural position for pauses (natural:at clause juncture or at the end of phrases) Frequency of disfluency markers (Hesitations, repetitions, self-corrections, and false starts) * Fluency is defined here only from temporal perspective. 4.0 Can produce smooth and continuous discourse with fairly high and even tempo. Very few unfilled pauses in unnatural/unlikely positions in an utterance occur. 3.0 Can produce relatively long discourse with fairly even tempo. Sometimes student make noticeable pauses in unnatural/unlikely position in an utterance and may be hesitant for searching patterns and expressions, repetition, self-correction. 2.0 Can keep going comprehensively, even though pausing for grammatical and lexical planning and repair in unnatural/ unlikely positions is evident, especially in longer stretches of production. 1.0 Can produce slow and often short discourse with many unfilled pauses in unnatural/unlikely positions of utterances. Utterances are hesitant and often contain repetitions, self-corrections, false starts. 0.0 No utterances. 2 Oral Presentation Intelligibility Key features rhythm, intonation, articulation 4.0 Rhythm, intonation, and articulation are easily comprehensible to everyone. 3.0 Rhythm, intonation, and articulation are reasonably comprehensible to competent listeners. L1 influence may be evident and occasional errors occur but do not cause comprehension problems. 2.0 Rhythm, intonation, and articulation require concentrated listening, but only an occasional intelligibility problem arises. 1.0 Rhythm, intonation, and articulation often impede comprehension, sometimes difficult to understand even with concentrated listening. 0.0 No utterances. 3 Oral Presentation Vocabulary Key features knowledge of English collocations, idiomatic expressions appropriateness and accuracy of word choices in a given context. 4.0 Has a good command of a broad range of language allowing his/her to select a formulation to express him/herself clearly in an appropriate style without having to restrict what he/she wants to say. 3.0 Has a sufficient range of language to be able to give clear descriptions, express viewpoints without much obvious lexical inaccuracies. 2.0 Has enough language to get by, with sufficient vocabulary to express him/herself with some lexical inaccuracies and circumlocutions. 1.0 Use basic sentence patterns with memorized phrases. Lexical inaccuracies may cause comprehension problems. 0.0 No utterances 4 Oral Presentation Organization/ coherence Key features main idea and support structure (opening, main body, summary) organizational device 4.0 Very good organization/coherence in oral discourse Smooth and attractive introduction and conclusion, good support for main idea(s) with adequate evidence. Clear organizational devices. 3.0 Clear main idea(s) with support with evidence. Introduction and conclusion are clear. Some deficiencies in organizational devices (discourse signals) 2.0 Main idea is identifiable but not adequately supported. Underlying structure is not sufficiently apparent. 1.0 Main idea is difficult to grasp. No apparent structure to follow. 0.0 No utterances 5 Oral Presentation Delivery Key features visual aid (optional)* body language eye contact volume of voice time use * For the oral presentation activity, the use of visual aid is not expected. 4.0 Presentation is delivered very effectively in a clear and sufficient volume of voice with effective body language and eye contact. (Visual aid is effectively used.) Given time is efficiently used. 3.0 Presentation is delivered clearly within a pre-set time (with the aid of visual materials.) Adequate body language and eye contact is given to the audience. 2.0 (Visual aid is not adequately used.) He/she slightly passes the pre-set time or the presentation is rather short. 1.0 Presentation is delivered with few body language and eye contact as if reading a manuscript. (Visual aids were not adequately used.) The volume of the voice may hinder comprehension. He/she passes pre-set time to a large extent or his/her presentation is too short. 0.0 No utterances 6 Group Discussion Fluency Key features Speaking rate Length of utterance unit Filled/unfilled pauses Natural/unnatural position for pauses (natural:at clause juncture or at the end of phrases) Frequency of disfluency markers (Hesitations, repetitions, self-corrections, and false starts) * Fluency is defined here only from temporal perspective. 4.0 Can produce smooth and continuous discourse with fairly high and even tempo. Very few unfilled pauses in unnatural/unlikely positions in an utterance occur. 3.0 Can produce relatively long discourse with fairly even tempo. Sometimes student make noticeable pauses in unnatural/unlikely position in an utterance and may be hesitant for searching patterns and expressions, repetition, self-correction. 2.0 Can keep going comprehensively, even though pausing for grammatical and lexical planning and repair in unnatural/ unlikely positions is evident, especially in longer stretches of production. 1.0 Can produce short discourse with many unfilled pauses in unnatural/unlikely positions of utterances. Utterances are sometimes hesitant and contain repetitions, self-corrections, false starts. 0.0 No utterances. 7 Group Discussion Intelligibility Key features rhythm, intonation, articulation 4.0 Rhythm, intonation, and articulation are easily comprehensible to everyone. 3.0 Rhythm, intonation, and articulation are reasonably comprehensible to competent listeners. L1 influence may be evident and occasional errors occur but do not cause comprehension problems. 2.0 Rhythm, intonation, and articulation require concentrated listening, but only an occasional intelligibility problem arises. 1.0 Rhythm, intonation, and articulation often impede comprehension, sometimes difficult to understand even with concentrated listening. 0.0 No utterances. 8 Presentation & Discussion Vocabulary Key features knowledge of English collocations, idiomatic expressions appropriateness and accuracy of word choices in a given context. 4.0 Has a good command of a broad range of language allowing his/her to select a formulation to express him/herself clearly in an appropriate style without having to restrict what he/she wants to say. 3.0 Has a sufficient range of language to be able to give clear descriptions, express viewpoints without much obvious lexical inaccuracies. 2.0 Has enough language to get by, with sufficient vocabulary to express him/herself with some lexical inaccuracies and circumlocutions. 1.0 Has basic language to convey rather simple ideas. comprehension problems. 0.0 No utterances 9 Lexical inaccuracies may cause Discussion Communicative skills/strategies Key features negotiate of meaning response eye contact gesture 4.0 Asks others to expand on views; shows ability to negotiate meaning; shows how own and others’ ideas are related; even, confident distribution of eye contact; use appropriate gestures; when listening use gestures and other cues to take the floor. 3.0 Responds to others; show agreement of disagreement to others’ opinions; eye contact maintained, but not used for turn taking, or emphasizing points; some spontaneous gestures. 2.0 Does not initiate interaction, produces monologue only, shows some turn taking, may say “I agree with you” but does not relate ideas in explanation; limited eye contact; often directed at one person when speaking; may look down or away if not speaking; gestures are rare 1.0 Mainly participate by active listening such as nodding, but not much by speaking. 0.0 Not measurable 10 Discussion Discussion content Key features quality of main ideas and support quality of contribution 4.0 Interesting, engaging content; a high level of personal response and rationalization. 3.0 Interesting and thoughtful; main ideas and examples are clearly distinguished. 2.0 Main points supported by details and examples; imaginative and interesting 1.0 Main points hard to identify; supporting details or examples were not adequate. 0.0 No utterances 11