NYME-SEK
Testing in ELT 1
Seminar material
Szücs Judit
Revised in 2010
1
Contents
1.
Course description
2.
Handouts
3.
Take-home assignment
2
1. Course description
Code: KANAN-8035 VIZSGAKURZUS
Title: Testing in ELT practice 1
Type of assessment: seminar grade
Credit: 1
Lecturer: Szücs Judit
Prerequisites: KANAN/KAEAN4022, KANAN/KAEAN6093, KANAN/KAEAN6094, or
KDEAN3022, KDEAN4093, KDEAN4094
Aims:
 To acquaint students with the principles and practice of language testing
 To prepare them to compile English language tests for various purposes
Content:
Language teaching and language testing.
Types of tests
Ways of measuring language competence
Validity and reliability
Backwash
Stages of test construction
Testing techniques and testing overall language ability
Testing writing
Testing reading
Assessment:
Portfolio (tasks in handout completed, notes, test samples etc., proof of working
with material)
20%
Written assignment
40%
End-term paper
40%
Bibliography:
Set titles:
Chapters/sections from:
Heaton, J.B. (1988) Writing English Language Tests Harlow: Longman
Heaton, J.B. (1990) Classroom Testing Harlow: Longman
Hughes, A.(1989) Testing for Language Teachers Cambridge: CUP
Recommended:
Alderson, Ch. and B. North (1991) Language Testing in the 1990s London: Macmillan
3
Alderson, J. Ch. and B. North (1991) Language Testing in the 1990s London: Macmillan
Alderson, J. Ch., K.J. Krahnke, and C.W. Stansfield (1987) Reviews of English language
proficiency tests. Washington D.C.: TESOL
Bachman, L.F. (1990) Fundamental Considerations in Language Testing Oxford: OUP
Bachman, L.F. and A.S. Palmer (1996) Language Testing in Practice. Oxford: OUP
Bárdos, J. (2002) Az idegen nyelvi mérés és értékelés elmélete és gyakorlata. Nemzeti
Tankönyvkiadó
Harrison, A. (1983) A Language Testing Handbook London: Macmillan
Henning, G.(1987) A Guide to Language Testing Cambridge, Mass.: Newbury House
Némethné Hock Ildikó (1998) Idegen nyelvi mérés és vizsgatechnika. Veszprémi Egyetemi
Kiadó
Oller, J.W. (1979) Language Tests at School Harlow:Longman
TESOL
Underhill, N. (1987) Testing Spoken Language Cambridge: CUP
Valette, R.M. (1967,1977) Modern Language Testing Harcourt Brace Jovanovich
4
2. Handouts
Kinds of tests
1. How often do you set progress tests?
2. What would you include in an end-of term test achievement test for you students?
3. How often do you give placement tests and selection tests?
4. Are proficiency tests ever necessary? If so, for what particular purpose
GROUP A
You work for a language school. Your job is to assign the new students to the following new
courses: General English Elementary, Intermediate and Post-Intermediate, Business English PreIntermediate and Intermediate and English for Nurses. What kind of tests would you use? What
would it contain (broadly)?
GROUP B
What kind of test would you use when trying to find out about what your new private student
needs? What would the test contain?
GROUP C
What kind of test would you give out at the end of the academic year? What would you like to
find out about with its help? What would the test contain?
GROUP C
What kind of test /tests would you use for screening applicants at the English Department of a
teacher training college? How long should it be, what should it contain (broadly)? What
information would you hope to get?
5
Kinds of testing
Seminar task:
Choose two tests and analyse them by answering the questions.
1. What is the purpose of the test?
2. Does it represent direct or indirect testing?
3. Are the items discrete point or integrative?
4. Can you identify objective and subjective items?
5. Is the term norm-referenced or criterion-referenced?
6. Does the test measure communicative abilities?
6
Validity – Reliability – Backwash
The relationship between validity and reliability
V -, R +
V +, R -
V +, R +
V -, R –
Analyse the following test items for validity, reliability, and possible backwash effect.
1
(The item is supposed to test vocabulary)
Select the option closest in meaning to the word underlined.
He began choking while eating the fish.
a) die
b) cough and vomit
c) be unable to breathe because of something in the windpipe
d) grow very angry
2
(The item is supposed to test Present Perfect)
Fill in the blank with the appropriate form of the verb:
I go to the cinema regularly, but I ............................. to the theatre for months.
3
(The item is supposed to test reading comprehension)
There are red and white stripes in our flag.
Our flag contains one ........... for each state.
a) stripe
b) star
4
(The item is supposed to test spelling.)
Rearrange the following letters to make English words.
PLEAP
SUHOE
ROLRY
IRACH
CELPA
EGURA
7
Test techniques and testing overall language abilities
Task 1
A) Match the following types of testing items and the examples:





1
rearrangement item
transformation item
multiple choice item
error recognition item
completion item
Tom ought not to ..... me your secret, but he did.
A tell
B having told
C be telling
2
'....... was Robert late last week?'
'Three times.'
A How much
B How often
C How many
D How long
3
Sun / is shining / brightly today /, isn't it?
A
B
C
D
4
'Won't I need my coat?'
'Well, you know how ..."
A it
B today
C warm
D have told
D is
5
Can you see ... sun shining through the clouds?
6
The best way to prepare ............................................................................... .
/is/yourself/past papers/timed practice/for the paper/to give/in doing/
7
I haven't written to you for a long time.
It's a long time ................. .
B) Discuss: What instructions would you write for them?
What do you think they test?
---------------------------------------------------------------------------------------------------Task 2
Design two or three items of each type.
--------------------------------------------------------------------------------------------------------Task 3
8
Brainstorm: What are the advantages and the disadvantages of multiple choice tests?
------------------------------------------------------------------------------------------------------Task 4
Cloze-tests
Do you know
- what cloze-tests are?
- how they are constructed?
- what they test?
When Walter was eight years old, in 1908, he used to work for an old lady who lived in his
village. Every day, before and after school, he would chop wood, light fires, and go shopping
for her. He was paid 5p a week, which was not very much even then.
This old lady kept so many cats that she did not know exactly how many. One day, one of
the cats was found dead in the garden, and the lady, who was very upset, asked Walter to
dig a grave and bury it. Walter did this and was given 10p. He couldn't help noticing that he
got twice as much for burying the cat as for a whole week's work. This made him think. He
had many friends and he offered them 2p for every dead cat they brought him.
By the time he left school, Walter had saved quite a lot of money. Later in his life he
became a very successful businessman.
Using the above text, create two cloze-tests. For the first, start deleting
from the first word of the third sentence, for the second, start it from another word. Compare
your tests, trying to identify differences and their implications on usability.
9
Testing writing
Task 1
Step 1
- What types of writing do some widely used commercial tests (state exams in Hungary, TOEFL,
Cambridge, Pitman, TELC, Euro etc. exams) include?
....................................................................................................................................................................
....................................................................................................................................................................
....................................................................................................................................................................
....................................................................................................................................................................
..................................
- What types of writing tasks could/should be included in a test for the following levels:
basic, intermediate, advanced?
....................................................................................................................................................................
....................................................................................................................................................................
....................................................................................................................................................................
....................................................................................................................................................................
..................................
- How far does the choice of tasks depend on the kind of test (e.g. proficiency, progress
achievement etc.)?
....................................................................................................................................................................
....................................................................................................................................................................
....................................................................................................................................................................
....................................................................................................................................................................
..................................
Task 2
Analyse the following test items for validity and reliability:
1. Testing Paragraph Writing
Use the following information to complete the sentence.
Use the additional words in brackets.
Omit the underlined words.
a)
some hydrogen ions are in contact with the copper plate
these hydrogen atoms take up electrons
the electrons are flowing to the copper plate
(that)
b)
the hydrogen ions become hydrogen atoms
(becoming)
c)
the removal of hydrogen ions reduces the positive potential
(and / from the solution / its)
Hydrogen ions .......................................................................... the copper
plate................................................... flowing to ....................................... plate,
............................................. hydrogen atoms, ...............................................the removal of hydrogen
ions ............................................................ reduces ......................................................positive potential.
10
2. Testing writing ability
‘Travel broadens the mind.’ (J. Smith).
Discuss.
Task 3 Scoring writing
Step 1: Read the 9-band marking scales in Hughes. Decide which one of you is going to use the
holistic and which the analytic one.
Step 2: Individually, read the enclosed compositions in Hughes and mark them according to your
scale.
------------------------------------------------------------------------------------------------------HW
With the help of scoring scales you can find in Hughes or other sources, work out a 5-band analytical
and a holistic scale to be used in Hungary.
11
Testing reading
 What is reading comprehension?
Activity
A) Make a list of the types of text that you have read in the last 24 hours, e.g. magazines, nonfiction textbooks. Write your list on the left-hand side.
magazines
textbooks
B)Now, write down the purposes in reading this particular text against each item on your list,
e.g. magazines - for general / specific information
(Adapted from Weir 1995 : 65-66)
 What specific skills are involved in reading?
 What is meant by intensive and extensive reading? What length of text is used for either? What is
their language difficulty level?
 How far does a scanning task differ from skimming?
 Close (and C-) tests - what do they really test?
 How would you establish the pass level in a reading test?
 Should dictionaries be allowed?
12
3. Take-home assignment
Testing in ELT 1 / practice
SKANAN7035
2010 Spring Term
TAKE-HOME ASSIGNMENT
Follow the steps below for your assignment (the steps need not be numbered).
 Choose a real or imaginary ELT testing situation. You are free to choose any purpose you prefer.
 In one paragraph, state the problem. Make sure you provide all the necessary information (e.g. title
and publication data of course book in the case of achievement testing) and mention all the relevant
considerations and constraints.
 Start working on the solution of the problem.
a. Write up the specification of the test to be used (you can follow the format in Examples 1 and
2 in Hughes).
b. Design ONE TASK (+ instruction) in full in the format you would use in the test.
Please bear in mind, that:
 You are not allowed to borrow text from existing printed or electronic publications (not even the set
textbook for the course) without supplying proper reference. If more than a phrase is used, you should
even give page reference.
 You can adapt existing test items, but not borrow them in their original format. Please make sure that
you state the origin.
 All the books, articles, etc. you consulted while working on your paper must be stated in the reference
section.


Word limit: 1500 ( + 10%) + appendix, if applicable.
All papers must be word processed (1.5 spaced, for electronic: Word 95 or above,.doc format –
no viruses, please!)

Papers are to be submitted in a hard copy (=printed) AND in an electronic version (mailed to
szjudit@btk.nyme.hu only)
Assessment to be used:
Content
Organization, referencing
Language (register, style)
TOTAL
15%
10%
15%
40%
Deadline
 8 May 12 pm for the electronic version
 10 May 12 am for hard copies
Szücs Judit
course tutor
21 February 2010
13