Test of English for Aviation
(TEA)
Henry Emery
Test development co-ordinator
Two main considerations:
• to produce a valid assessment procedure
that reflects candidates’ proficiency
according to the ICAO Rating Scale
• to produce an objective, standardised test
format that can be delivered in the same
way by different examiners
TEA
• is based on the recommendations of ICAO
Document 9835
• tests plain English in a work-related context
(Section 2.7.3, ICAO Doc 9835)
• tests language proficiency in unexpected
situations (Section 2.7.4, ICAO Doc 9835)
• is not a test of radio-telephony phraseology
– TEA is one possible model
– Standard phraseology still needs to be
evaluated
• assesses appropriacy of language use, not
procedure
TEA
• is designed to elicit language that is
assessable according to the ICAO Rating
Scale
• is conducted in the form of a direct and semidirect one-to-one (candidate:examiner)
interview
• consists of three sections and lasts between
16 - 20 minutes
• is suitable for both pilots and controllers
TEA
• Part 1 - Interview
• Part 2 - Interactive comprehension
• Part 3 - Description and discussion
Part 1: Interview
• Direct
• Common, concrete and work-related
topics
• Set questions on familiar topics, based
on aviation background
• 3-4 minutes
• Warmer
• Initial impressions
Part 2: Interactive comprehension
• Semi-direct
• no eye-contact, no facial cues
• Candidates respond to international
speakers
• Simulated real-life communication
• 6-8 minutes
• limited examiner role
Part 2: Interactive comprehension
‘Live’ Comprehension allows for
• demonstration of discourse
management strategies
• demonstration of proficiency across
all six language profiles
Part 2a
• Six aviation-specific recordings
• Non-routine and emergency
situations
• Candidates report back to the
examiner on what they hear
• Authentic text
• Time limit
Part 2a: example statements
• “It’s possible we won’t have brakes on
touchdown. Can we have the longest
runway possible?”
• “We’ve lost hydraulic pressure. We’ve
got limited control of the aircraft”
Part 2b
•
•
•
•
•
•
•
•
Four recordings in a general situation
Problem-solving
Non-routine and emergency situations
Candidate is required to ask questions and
give advice
Time limit
Avoids reference to operational procedure
Human, environmental, health, technical
Example (unrated)
Part 2b: example statements
• “There’s a woman crying
She says she’s lost her passport” (human)
• “I can’t steer the vehicle
The ice is too thick” (environmental)
• “Ahh, this really hurts
I think I’ve broken my ankle” (health)
• “I can’t reach the controls
I can’t open the door” (technical)
“I can’t reach the controls
I can’t open the door” (technical)
PILOT LOCKED OUT AFTER TOILET TRIP
•
TORONTO Passengers on a Canadian plane had an
unsettling in-flight experience after the pilot found
himself locked out of the cockpit after a trip to the toilet.
The pilot was seen banging on the door and talking to his
first officer on an internal phone. Crew members were
forced to take the door off its hinges.
THE INDEPENDENT Friday 01st September 2006
Part 2b: example statements
• “There’s a woman crying
She says she’s lost her passport” (human)
• “I can’t steer the vehicle
The ice is too thick” (environmental)
• “Ahh, this really hurts
I think I’ve broken my ankle” (health)
• “I can’t reach the controls
I can’t open the door” (technical)
Part 3: Description and discussion
•
•
•
•
Direct
A minimum of 3 aviation-related images
Routine and emergency situations
Candidates describe the image and then
respond to graded follow-up prompts
delivered by the examiner
• Examiner flexibility - ‘high-end’ discriminators
I’d like to ask you some general questions
related to weather.
• (Compare) How do different
weather conditions affect aviation
where you work?
• (Describe) How would you describe
perfect flying conditions?
• (Consider) How weather will affect
aviation in the future?
I’d like to ask you some general questions
related to the problems associated with fire.
• (Explain) What problems can fire cause
for aircraft?
• (Describe) What equipment or methods
can be used to control fire?
• (Suggest) Why is fire so dangerous for
humans?
Language assessment
• Candidates are awarded a band score
most appropriate to performance in
each language profile
• Candidates must demonstrate
proficiency level 4 in all language
profiles (Section 2.9, ICAO manual)
• Rating - Lowest band score (Section
2.8.4, ICAO manual)
Test administration
• There is no previous contact between
examiners and candidates
• The same person performs the role of
examiner and rater
• All examinations are recorded and archived
for future reference
• 10% of test recordings are assessed by a
second rater to ensure reliability of results
Test administration
• All examiner / raters are native speaking,
diploma- qualified TESOL teachers with
aviation teaching experience
• All examiners receive training in rating and
exam procedure prior to operating as a TEA
examiner / rater
• All examiners adhere to the ILTA code of
ethics
Test security
• Test materials are kept under lock and
key
• Confidentiality of speech samples and
results
• Identification is checked before the start
of the test
Test security
• All candidates are photographed
• Certificates include the photograph and
biographical data of the candidates
• Counterfeit-proof certificates
• Centralised certification
Test Development and Maintenance
• Academic consultation with language testing
researchers
• Operational (pilot/ATC instructors)
consultation to ensure test validity
• Practical consultation from experienced
examiners and teachers
• Feedback from examiners and candidates
Test Development and Maintenance
• On-going validation using Observation
Checklists and comparative analysis
• Pre-testing
• Pre-tested items introduced into live test
material
• College Rater Support and Monitoring
Programme to maintain reliability of results
Questions, comments and feedback
henry@maycoll.co.uk