ENLTA
European Network
for Language Testing
and Assessment
EALTA
European Association
for Language Testing
and Assessment
ENLTA
• Funded by European Commission to
create EALTA - a network of individual
language testers
• Two years, December 2003 – December
2005
• 14 partner institutions
• 8 activities in the Work Plan
Activities
1.
2.
3.
4.
5.
6.
7.
8.
The organisation of EALTA
Analysis of training needs
1st Annual Conference
Survey of assessment policy and practice
Develop and pilot a training model
Develop a code of good practice
2nd Annual Conference
Identify a basic training resource
Presenters
• Activity 4: Survey of assessment policy
and practice in Europe
• Gudrun Erickson
• Sauli Takala
• Dianne Wall
Presenters
• Activity 5: Develop and pilot a training
model
• Ülle Türk
• Melanie Ellis
Presenters
• Activity 8: Identify a basic training resource
• Melanie Ellis
E N L T A – Activity 4
Survey of Policy and Practice
• Gudrun Erickson, Jan-Eric Gustafsson &
Cecilia Nihlén, Sweden
• Sauli Takala, Finland
• Barbara Czarnecka-Cicha, Poland
• Tania Horak & Dianne Wall, UK
AIMS
• Links to, and short comments on previous studies of
assessment cultures and practices (EALTA website)
• Links to, and short comments on official language
testing sites in different European countries (EALTA
website)
• Reports on [some] European students’ and teachers’
thoughts about language assessment practices
(EALTA website)
Review of literature
The purpose is to provide a list of readings which deal
with pupils’ views of assessment, tests and
examinations.
We are mainly interested in empirical studies but also
reviews, essays and position statements are of
interest.
We have consulted Finnish (Linda) and Swedish
(Libris) data bases as well as ERIC, Language and
Linguistics Behavior Abstracts, and a couple of other
sources.
•Kärkkäinen, K. & Takala, S. (1978) A feasibility study
of incorporating a structures test in the matriculation
examination. Reports from the Institute for Educational
Research, University of Jyväskylä, 290/1978.
• IEA: Language Education Study (Ari Huhta et al.)
•Westhpal, P.B., Wacha, H. & Rhodes, P. (2002)
Assessing oral performance in the secondary
classroom. French Review, v. 75, n 3, 560-569.
• Kenyon, D. M. & Malabonga, V. (2001) Comparing
examinee attitudes toward computer-assisted and
other oral proficiency assessments. Language
Learning and Technology, v 5, n2, 60-93.
• Barnes, A., Hunt, M. & Powell, B. (1999) Dictionary
use in the teaching and examining of MFLs at GCSE.
Language Learning Journal, 19, 19-27.
• Powell, B., Barnes, A. & Graham, S. (1996) Teachers´
views of target language testing. Language Learning
Journal, 14, 3-9.
• Alderson, J.C. & Clapham, C. (1995) Assessing
student performance in the ESL classroom. TESOL
Quarterly, 29, 1, 184-187.
• Sjöberg, A. (2002) Functionality of language skills in
occupational English: the point of view of language
users, language training and language testing. PhD
thesis, University of Oulu.
In order to be able to cover all the relevant literature we
ask for your assistance.
Please send information about electronic data bases that
we might search.
References following the model provided above are most
welcome. To make the material more easily accessible,
please provide a translation into English of the title (when
needed). Also, whenever possible, provide a brief abstract
or summary in English. If the whole publication is
available, it can be sent to the following address.
sjtakala@hotmail.com
Address: Luokotie 2
40950 Muurame
Finland
We will provide a register of the references to the EALTA
membership and search for ways to analyse them an
publish the results.
Links to official language testing
resources in Europe
We have drawn up a list of sites we have
found. We would be very grateful if EALTA
members sent us details of any other sites
they feel would be of interest and
relevance. If your own organisation is
missing from the list on the next pages,
please send information to
d.wall@lancaster.ac.uk
Links to official language testing
resources in Europe
International:
ETS Europe
International Baccalaureate Organisation
Belgium
Centre for Language and Migration
University of Antwerp Language Test Centre
Links to official language testing
resources in Europe
Czech Republic
CERMAT (School-leaving examinations)
STANAG Examination Centre
Language School State Exams
Finland
Finnish Matriculation Examination Board
Links to official language testing
resources in Europe
France
Franc-Parler
Germany
DAAD
TestDaF
Thuringian Ministry of Education
Unicert
Links to official language testing
resources in Europe
Greece
Greek State Certificate
Hellenic American Union
Hyphen
Links to official language testing
resources in Europe
Hungary
Hungarian Accreditation Board for Foreign
Language Examinations
Hungarian Association of Language Examiners
and Measurement Specialists
Hungarian Exams Reform Teacher Support Project
Institute of International Education, European
Office
+ about 20 links to exams approved by OKI
Links to official language testing
resources in Europe
Italy
Trinity College Italy
University for Foreigners of Siena
University of Rome “Roma Tre”
Latvia
Centre for Curriculum Development and
Examinations
Links to official language testing
resources in Europe
Lithuania
National Examinations Centre
Public Service Language Centre
Luxembourg
Centre de Langues Luxembourg
Links to official language testing
resources in Europe
The Netherlands
Bureau ICE
CITO
Norway
AKSIS/ University of Bergen
Directorate for Primary and Secondary
Schools
Links to official language testing
resources in Europe
Poland
Central Examinations Commission
Russia
Federal Institute for Educational
Measurement
Federal Testing Centre
Unified State Examinations
Links to official language testing
resources in Europe
Slovenia
Centre for Slovene as a Second/Foreign
Language
National Examinations Centre
Spain
Escuela Oficial de Idiomas de Avilés
Instituto Cervantes
Links to official language testing
resources in Europe
Sweden
Göteborg University, Dept of Education
Turkey
OSYM – Student Selection and
Development Centre
Links to official language testing
resources in Europe
United Kingdom
Cambridge ESOL
City and Guilds Pitman Qualifications
DfES – Information on ESOL qualifications
Dave’s ESL Café Assessment Forum
Federation of Awarding Bodies
Institute of Linguists
Learning and Skills Council
Links to official language testing
resources in Europe
United Kingdom (cont.)
QCA – Qualifications and Curriculum
Authority
Scottish Qualifications Authority
Test of English for Educational Purposes
Trinity College London
Survey: Student Questionnaire
Piloted in 3 countries (Nov. 2004)
Administered in Dec. 2004 – March 2005
Referring to 1st Foreign Language
If possible, answers in English
Open-ended questions
What is good language assessment?
What is bad language assessment?
Why?
Other comments?
Likert scales
Attitudes to and feelings about testing, assessment
and achievement
Areas emphasized in testing and assessment
Distinction made between exams/formal tests and
assessment during lessons
Students
1, 373 students in ten European countries
Bulgaria, Estonia, Finland, Hungary, Norway,
Poland, Slovenia, Spain (Catalunya) , Sweden,
the UK
End of compulsory school
Survey: Teacher Questionnaire
Comments on
the whys, whats and hows of language
testing and assessment
+
Likert scales
62 responses
Some preliminary results
• Agreement between students and teachers
about what is most frequently tested /
assessed:
– grammar
– words & phrases
– reading comprehension
– written production
• Agreement that assessment of oral
proficiency is much less frequent, especially
in formal testing situations.
Students’ perceptions
Examples of positive features in tests
• Variety
• Communication
• “Usefulness”
• Pedagogical potential
• Clarity
• Fairness
• Enough time
Students’ perceptions
Examples of negative features
• Too much focus on single aspects of
language, in particular grammar
• Too little focus on communication,
especially speaking
• Too little focus on active language use
• Stress
Examples of students’ comments
“A good language test is a test whiff a lot
of diferent parts. Not just gramma, it is to
much. The best test is an essaytest,
becouse there you test a lot of different
things at the same time.”
“A good test is the one which enables you
to learn and you can see (find out) what
you know (can). A bad test makes you
nothing to learn. You do not learn
anything by doing it.”
Examples of students’ comments
“In my opinion, good language assessment is assessing
students for all the time - during classes, not only during
tests. For me, writing texts, grammar or vocabulary is
the same importaint as speaking, reading or listening. A
good language test is checking all language abilities.
There are many people who are very good at grammer,
but they have some problems with speaking; there are
also people who are good at speaking but they are not
good at grammer, so they can’t improve their english at
such ”bad” tests. I think the best way to teach and to
learn english is to encourage and to be encouraged, to
interest and to be interested in language. Some tasks,
projects and films(movies) make students encouraged
to learn the language. ”
Examples of students’ comments
“I like language tests and assessments because
they show how much I know about the
language. – I think speaking tests are better
than writing because if you listen to somebody
speak you hear his/her pronounciation as well
and also how he/she expresses himself/herself
and of course how he/she knows grammar.”
“BAD:-fast, I mean not enough time, so
generating stress-veeery narrow domain-only
ONE answer is correct.”
Examples of students’ comments
“This ENLTA/student questionaire doesen’t like me either
because probably it’s never read by someone other than
me so basicly it wastes our lesson time and don’t do any
good for me or my classmates”
“Thank you for taking time to lisen one my toughts.”
“This was nice questionnaire!”
“Good luck with the reasearch!”
ENLTA Activity 5
Survey of training needs:
Develop and pilot a model for training events
and modes of delivery
The team
•
•
•
•
•
•
•
•
Györgyi Együd (Hungary)*¤#
Melanie Ellis (Poland)*#
Neus Figueras (Spain)+
Sara Gysen (Belgium)
Angela Hasselgreen (Norway)*
Günter Nold (Germany)*
Ülle Türk (Estonia)*#
Norman Verhelst (the Netherlands) (Classical Test
Theory)#
•
•
•
•
* prepared materials
+ reviewed materials
¤ organised the training event
# delivered training
Tasks
• Plan training events and modes of delivery
• Design pilot training events and time schedule of
delivery
• Deliver pilot training event
• Report to 2nd conference
• Two face-to-face meetings:
– Kranjska Gora (May 2004): initial planning
– Frankfurt (September 2004): detailed planning
Initial decisions
• Two separate events:
– Introduction to classical test theory – Norman Verhelst
– Introduction to testing and assessment – the rest of the
team
• A three-day event to take place in Szeged,
Hungary in March 2005
• The target group: foreign language teachers who
have had no previous training in testing and
assessment
• Classroom assessment, self-assessment and ELP
included
Day One
• 09.30–11.30 Introduction to testing and
other forms of assessment
• 12.00–13.30 Introduction to assessing
receptive skills
• 14.30–16.00 Assessing receptive skills:
Test formats, task types, principles of
scoring, issues related to scoring
• 16.30–18.00 Classroom assessment
including ELP – focus on listening
Day Two
• 09.00–10.30 Introduction to testing
productive skills
• 11.00–12.30 Testing writing: focus on
tasks
• 13.30–15.00 Testing writing: focus on
assessment
• 15.30–17.00 Classroom assessment of
writing
Day Three
• 09.00–10.30 Testing speaking: what and
how to test
• 11.00–12.45 Testing speaking:
assessment
• 13.30–15.00 Classroom assessment of
speaking
Timeline
•
•
•
•
•
•
First draft
Feedback
Second draft
Feedback from Neus
Final version
Training event in Szeged
30 November
10 January
25 January
10 February
01 March
10–12 March
Lessons learnt
•
•
•
•
22 hours too little:
– More time for reading and listening needed
– No time for classroom assessment
– ELP – a separate issue
Possible solutions:
– Exclude classroom assessment
– Increase the number of hours
A five-day event more realistic
– Delivered as one event
– Two modules:
(1)General principles and testing receptive skills
(2)Testing productive skills
Local needs
– Core materials supplemented by local materials
Activity 8: Resources
• Melanie Ellis
– Foreign Language Teacher Training College,
Zabrze, Poland
• Laurence Kane
– University of Dortmund, Institute for English
and American Studies
• Milena Ivanova Grigorova
– New Bulgarian University
Aims of the evaluation
• To evaluate the pilot materials in use
• To evaluate the training event
• To gather ideas for resources teachers
need to support their learning about
testing and assessment
How did we carry out the
evaluation?
• Observation and Feedback sheets
– timing, order of activities, choice of interaction
– content evaluation: comments, suggestions,
questions
– overall impressions: two perspectives- the
observer and the trainer
• Feedback sheets on each session for
participants
• End of course feedback sheet
Qualitative evaluation
• Focus group discussions at end of each
day
• Introductory session task and final
reflection task
By the end of the course I’d like to
know more about...
have learnt...
be able to...
End of course feedback
• I enjoyed the course
5 (13) 4 ( 5)
• I found the course useful
5 (10) 4 (8)
• I learned new things
5 (6)
4 (7)
3 (5)
• The classes were well-organised
5 (11) 4 (6)
3 (1)
ENLTA Activity 8
As part of Activity 8 we are preparing a bibliography for
teachers who want to develop their knowledge of and skills
in testing. The result will be available on the EALTA wesite.
Please help us by listing any book, CD, tape, video,
computer program or website you have used and would
recommend.
Send your response to
melanie@ellis.pol.pl. and put 'Bibliography' in the Subject
line
Thanks