Matakuliah : G0444 / MATERIAL DESIGN and TESTING
Tahun
: 2009
Pertemuan 19
I. READING TESTS
Question types of Reading Comprehension Tests :
1. True or False
2. Multiple Choice
3. Open Ended Choice
4. Information Transfer
5. Cloze
3
ADVANTAGES & DISADVANTAGES
1. TRUE OR FALSE:
a. it is easy for the teacher to think of good questions which
cover the main points of the text.
b. The questions only test comprehension ( no writing
involved), and are easy to mark.
c. the students can guess the answer: they have a 50% chance
of being right each time. (Therefore, to reduce the chance of
guessing, give three choices: true, false, don’t know)
4
2. MULTIPLE CHOICE :
a. it is easy to mark, so it is widely used in formal tests
b. test only comprehension, but there is only a small chance of
guessing the right answer.
c. it is difficult and more expensive to prepare. It has to be
written by specially trained item writers and to be pre-tested
before use in formal examination. Each item has to be
rigorously edited to ensure that:
5
- There is no superfluous information in the stem.
- The spelling, grammar and punctuation are correct
- The language is concise and at an appropriate level for
candidates
- Enough information has been given to answer the
question
- There is only one unequivocally correct answer
- The distracters are wrong but plausible and
discriminate at the right level
- The responses are homogenous, of equal length
and mutually exclusive and the item is
appropriate for the test.
6
3. OPEN - ENDED QUESTIONS :
a. it is easy to write but may be difficult to mark.
4. INFORMATION TRANSFER:
a. it is a good way of testing comprehension of the main points of a text (
because instead of answering a question, the students has to record
information from the text in a different form, e.g. by completing a table,
labeling a picture, drawing a diagram)
b. it may not show whether the student has understood the text completely.
7
5. CLOZE :
a. it is easy to construct and easily scored if the exact word scoring
procedure is adopted.
b. it is claimed to be valid indicators of overall language proficiency.
c. it produces more successful tests of syntax and lexis than of reading
comprehension tests in general.
8
II. LISTENING COMPREHENSION TESTS

Same kind of questions can be used to test Listening comprehension as for
Reading comprehension. The text is read out by the teacher as a Listening
test.
 DIFFERENCES from Reading Test :
a. the questions should be simpler and not too many.
b. in reading, students can keep referring back to the text, but in listening
they have to retain what they hear in their minds. So, the text itself should
also be fairly short.
c. in listening, we can test students’ ability to recognize
words and
phrases which would present no problem in reading. So, we could include
questions which can be answered directly from the text.
d. if the main aim is to test listening skill, we should avoid giving students
too much to write.
9
QUESTION TYPES
TESTING EXTENSIVE LISTENING SKILLS :
1. Multiple Choice Questions (MCQ)
2. Short Answer Questions (SAQs)
3. Information Transfer Techniques
10
TESTING INTENSIVE LISTENING :
E.g. Dictation
Advantages & Disadvantages:
(1). it gives results similar to those obtained from Cloze tests.
(2). in predicting the overall ability, it has the advantage of
involving listening ability.
(3). It is easy to create and to administer.
(4). It is certainly not easy to score.
(5). If the dictation is not recorded on tape, the test will be less
reliable, as there will be differences in, e.g., the speed of delivery of
the text to different audiences.
11
LISTENING SKILLS :
1. MACRO - SKILLS include:




Listening for specific information
Obtaining gist of what is being said.
Following directions
Following instructions
2. MICRO – SKILLS include :
 Interpretation of intonation patterns ( recognition of sarcasm, etc)
 Recognition of function of structures ( such as interrogative as request,
for example : could you pass the salt ?).
12
III. TESTING WRITING
 It is very common for “writing“ test to focus on grammar. This is
because it is easy to test grammar – there are answers, marking is easy,
etc.
 If tests only focus on grammar, they will not show how well students
can write in English to express meaning.
 If we want to encourage students to develop writing skills, then it is
important to give tests in which students have to express meaning in
written sentences, not merely write correct grammatical forms.
13
TWO DIFFERENT APPROACHES
for assessing writing ability
1.
INDIRECT METHODS :
Writing can be divided into discrete levels : grammar, vocabulary,
spelling and punctuation. These elements can be tested separately by the
use of objective tests.
2. DIRECT METHODS :
With a more integrative and direct approach to the testing of writing, we
can incorporate items which test a candidate’s ability to perform certain
of the functional tasks required in the performance of duties in the target
situation (essay type of tests). For example : for doctors in hospital this
might involve writing a letter to another doctor ( local GP) about a
patient’s case based on the printed forms/notes.
14
MARKING FREE WRITING
Free writing tests have two disadvantages :
(1). Because students are free to write what they like, they are
likely to make many mistakes of different kinds. So the tests
will tell us very little about what they can and cannot do, or
how much progress they have made.
(2). Because there is no single correct answer, free writing tests
are difficult to mark precisely and marking takes a long time.
15
Different ways of marking them :
(1). Negative marking : We count all the mistakes the student has made,
and subtract, say, half a mark for each mistake.
(2). Positive marking : We give, say, three marks for each sentence. If a
student has written the sentence more or less correctly, they are given three
marks; if they have made some mistakes but the sentence can be easily
understood, they are given two marks; if the sentence is very hard to
understand, they are given one mark.
 Notes :
- Positive marking gives more emphasis to the content and meaning the
student is trying to express;
- By giving a positive mark, we reward students for what they can do rather
than “punish” them for what they cannot do;
- Obviously, this will have a better effect on their attitude to learning.
16
IV. TESTING ORAL ABILITY

The objective of teaching spoken language is the development of the
ability to interact successfully in that language. This involves
comprehension as well as production.

If we want to encourage students to speak, we should give oral tests
from time to time; otherwise, students will always regard speaking as less
“serious” than the other skills.

Obviously, it is very difficult to test speaking, especially with large
classes. Whereas with listening, reading and writing students can all be
tested at the same time, with speaking each student ( or pair of students)
must be tested in turn.
17
TWO MAIN WAYS OF TESTING SPEAKING ABILITY
1. CONTINUOUS ASSESSMENT
The teacher can either give a mark for general willingness and ability to
speak in all the lessons, or else note down marks for certain oral
activities done in class ( e.g. role play ). This system will only be
successful if the teacher tells the students that they are being given marks
for speaking.
18
2.
SHORT ORAL TESTS
The student is expected to give a short talk on a topic which he has either
been asked to prepare beforehand or has been informed of shortly before
the test. To organize for a large class:
a. give the whole class a set of general topics to prepare a few
before (e.g. talk about your family, a sport you enjoy).
weeks
b. over 2-3 lessons, the teacher calls each students out in turn
and
quietly tests them on one of the topics. Each test lasts 30- 60 seconds. The
students do not know which topic they will be
asked to speak about.
c. teachers can give a mark immediately. To help them do this,
can use a “marking grid”.
19
they
OTHER GENERAL FORMATS
of testing ORAL ABILITY
1.
2.
3.
4.
Interview
Information transfer: description of a picture sequence
Information transfer: questions on a single picture
Interaction tasks: - information gap student – student
- information gap student - examiner
5.
6.
Role play
Response to tape recordings
20
ELICITATION TECHNIQUES
The appropriateness of each technique will depend upon the
specification of the oral test. Most of them can be fitted easily within an
interview framework.
(1). Questions and requests for information: yes/no questions should
generally be avoided. Various functions can be elicited through requests of
the kind : “Can you explain to me how/why…….”and “Can you tell me what you
think of ……”
(2). Pictures : single pictures are particularly useful for eliciting descriptions.
Series of pictures (or video sequences) form a natural basis for narration.
(3). Role play : candidates can be asked to assume a role in a particular
situation. This allows the ready elicitation of other language functions.
e.g.: you want your mother (played by the tester) to increase your pocket money.
She is resistant to the idea. Try to make her change her mind.
21
(4). Interpreting : simple interpreting tasks can test both production and
comprehension in a controlled way.
(5). Discussion : can be a discussion of a topic or in order to come to a
decision, such as : “there is too much sport on television”. Discuss the
advantages and disadvantages .
(6). Tape recorded stimuli
(7). Imitation: candidates hear a series of sentences, each of which they have
to repeat in turn.
22
ADVICE ON
PLANNING AND CONDUCTING ORAL TESTS
1. Make the oral test as long as is feasible.
2. Include as wide a sample of specified content as is possible in the time
available.
3. Plan the test carefully.
4. Give the student as many “fresh starts” as possible. If possible, more
than 1 format should be used.
5. Select interviewers carefully and train them.
6. Use a second tester for interviews.
7. Set only tasks and topics that would be expected to cause student no
difficulty in their own language.
8. Carry out the interview in a quiet room with good acoustics.
9. Put students at their ease.
10. Collect enough relevant information.
11. Do not talk too much.
23
SCORING and interacting during ORAL TESTS
Scorer reliability in oral tests is helped not only by global assessment score and
analytic profiles but also by separating the role of scorer ( or examiner)
from the role of interlocutor ( the examiner who guides and provokes
conversation).
By doing this, it will allow the scorers to observe and assess, free from the
responsibility of keeping the interaction with the candidate or candidates
going.
In many tests of Speaking, students are now put in pairs or groups for certain
tasks since it is felt that this will ensure genuine interaction and will help to
relax students in a way that interlocutor – candidate interaction might not.
Bina Nusantara