Annotated Bibliography and Literature Survey:
Assessment of Language Proficiency
in the Health and Social Services Sector
Research Report
Presented by Juliane Bertrand
November 24, 2005
Terms of Reference
Survey the following:
1) ESL proficiency tests for adults working in a
professional environment, with emphasis on health
and social services workers
2) Materials generated by each such test
3) Studies on ESL proficiency assessment
Main ESL Proficiency Tests
• Administered directly by the design agency
- Test of English as a Foreign Language (TOEFL)
- Michigan English Language Assessment Battery (MELAB)
- Test of English for International Communication (TOEIC)
- ETS Test of Professional English (ETS TOPE)
• Administered by the client institution
- Institutional Testing Program (TOEFL ITP)
- Speaking Proficiency English Assessment Kit (SPEAK)
- Secondary Level English Proficiency Test (SLEP)
- Michigan Test of English Language Proficiency (MTELP)
- Comprehensive English Language Test (CELT)
Description of ESL Proficiency Tests
Test
Duration
Type of questions
Multiple
choice
Short
answer
Skills assessed
Composition
and/or
interview
Oral
comprehension
Written
comprehension
Written
production
X
(written)
X
X
X
(TWE)
X
X
X
X
X
X
TOEFL
paper
3.20 hrs
X
TOEIC
2.30 hrs
X
MELAB
3.30 hrs
X
TOEFL
ITP
1.55 hrs
X
X
X
SLEP
1.20 hrs
X
X
X
SPEAK
20 min
CELT
2.15 hrs
MTELP
1.30 hrs
X
(both)
X
(oral)
Oral
production
X
School
Workplace
X
*
X
X
X
X
X
X
X
X
X
Environment
X
X
X
X
X
Area of
health
and
social
services
Conditions for
Language Proficiency Testing
Test
Type of room
Staff training
Cost
TOEFL
Regional centres
N/A
Test taker: US$140
TOEIC
At the institution
Standard room
or computer lab
TOEIC staff, or TOEIC-trained institutional staff
Institution:
$45 + taxes per test taker
TOEFL
ITP
At the institution
Standard room
Instruction manual supplied by ETS
Marking by ETS representatives
Institution:
$42.50 + taxes per test taker
MELAB
At Ann Harbour
N/A
Test taker: US$120
SLEP
At the institution
Standard room
Starter kit:: US$235
10 question booklets: US$85
100 answer sheets: US$85
CELT
At the institution
Standard room
Examiner's kit: $45
Cassette: $45
10 question booklets: $120.50
100 answer sheets: $55
Other costs per section: $24
MTELP
At the institution
Standard room
N/A
Starter kit: US$165
20 question booklets: US$75
100 answer sheets: US$10
Strengths / Weaknesses
Test
Strong Points
Weak Points
TOEFL
paper
- Standardized test
- International reputation
- Does not assess oral production
- Intended for the school environment
- Too general (not geared to health and social services)
TOEIC
- Standardized test
- International reputation
- Designed to assess workers
- Does not assess oral production
- Too general (not geared to health and social services)
MELAB
- Standardized test
- Assesses the four language skills
- Not recognized in the health field
- Administered in Michigan (USA)
TOEFL ITP/ - Can be administered locally
SLEP/
CELT/
MTELP
SPEAK
- Can be administered locally
ETS TOPE - Assesses the four language skills
- Referenced to typical workplace tasks
- Does not assess oral or written production
- Intended for the school environment
- Too general (not geared to health and social services)
- Does not assess oral production
- Intended for the school environment
- Too general (not geared to health and social services)
- Not tailored specifically to the health and social services
sector
- Administered in China
Canadian Tests
• Canadian Test of English for Scholars and Trainees
(CanTEST)
Positives
- Assesses the four language skills
- Uses authentic material
Negatives
- Administered only in Toronto, Ottawa, Saskatoon, Winnipeg and
Halifax
- Not geared to the needs of health and social services workers
Canadian Tests
• Canadian English Language Benchmark Assessment for Nurses
(CELBAN)
Positives
- Assesses the four language skills
- Referenced to nursing tasks
Negatives
- Administered only in Vancouver, Toronto, Edmonton and
Winnipeg
Canadian Tests
• CanTEST
• CELBAN
• Cost: about $200, at testtaker’s expense
+ travel to a city where the
test is administered
• Cost: $300 plus taxes, at
test-taker’s expense
+ travel to a city where the
test is administered
• Duration: 3 hrs
• Duration: 3 hrs 30 min
Other Assessment Instruments
• Canadian Language Benchmarks (CLB)
- Set standards for the four language skills
- Centred on the test-taker’s social and professional activities
- Lay the ground for designing tests for specific job sectors
(e.g. CELBAN)
• Portfolio
- Tracks the learner’s progress
- Avoids psychological factors that could affect test performance
- Requires ongoing assessment
Materials Generated by these Tests
• Works on theory and practice
- More than 70 research reports published by ETS
(e.g. TOEFL, TOEIC, SLEP, TSE) since 1977
- Numerous CCLB documents describing the
theoretical bases of Canadian language standards,
the characteristics of those standards, and resulting
ideas for activities
- Technical manuals on administering language
proficiency tests; charts to help interpret test results
Materials Generated by these Tests
• Validity studies
- Frequently re-assessed for standardized tests
- Difficult to produce for locally administered tests
given widely varying test management procedures
- CELBAN = a first study suggests a positive link
between success with CELBAN and withTOEFL
(CCLB, 2004)
Materials Generated by these Tests
• Testimonials from institutions using these tests
- Overall satisfaction with TOEFL and TOEIC
- TOEFL unsatisfactory for nurses
- CELBAN = a study on nurse and administrator
satisfaction should be undertaken in 2006
Studies on ESL Proficiency Assessment
• Background
- Prior to the communicative approach, language
proficiency was the sole focus of assessment.
- Integrated test introduced
- Leading works:
Lado, Robert (1961). Language Testing, London, Longmans,
xxiv + 390 p.
Valette, R. M. (1967). Modern Language Testing: A Handbook,
New York, Harcourt, Brace & World.
Cooper, R. L. (1968). “An elaborated language testing model,”
in Problems in Foreign Language Testing: Proceedings of
a Conference. Language Learning, 18, p. 57-73.
Spolsky, B. (1978). Approaches to Language Testing: Advances
in Language Testing, Series 2, Arlington, VA, Center for Applied
Linguistics.
Studies on ESL Proficiency Assessment
• Present trends
- With the communicative approach, assessment covers all language
skills.
- Introduction of alternatives to the standardized test and of tests
referenced to quasi-authentic tasks.
- Leading works:
Canale, M. and M. Swain (1980). “Theoretical bases of communicative
approaches to second language teaching and testing,” Applied Linguistics, 1,
p. 1-47.
Bachman, L. (1990). Fundamental Considerations in Language Testing, Oxford,
Oxford University Press.
Celce-Murcia, Z. Dörnyei, S. Thurrell (1995). Communicative competence:
A pedagogically motivated model with content specifications. Issues of Applied
Linguistics, 6 (2), p. 5-35.
Milanovic, Michael and Nick Saville (Eds.) (1996). Performance Testing,
Cognition and Assessment. Selected papers from the 15th Language Testing
Research Colloquium, Cambridge and Arnhem, Cambridge, Cambridge
University Press, coll. “ Studies in Language Testing 3,” 314 p.
Conclusion
• None of the tests surveyed can be used to assess proficiency in
English as a second language for all job categories in the health
and social services sector.
• CELBAN, designed with reference to CLB, seems to be a model
to follow. It is consistent with current language acquisition
theories (e.g. emphasis on language objectives rather than
specific grammatical constructs and on the use of authentic
tasks).