L540: Test Critique
Before we can look at different types of alternative ESL/EFL assessments, we have to take a step back
and look at standardized tests to understand what they can or cannot do. By understanding the
possibilities and limitations of standardized language tests, we may be able to identify why alternatives
to such tests are necessary in order for us to assess language and content in tandem.
Attached is a sample of a test critique.
Assignment Instructions:
Choose a specific test to evaluate. Use the following elements to evaluate the test:
Evaluation Elements
Points
a. Test purpose and Use:
What are the intended uses of test? Who is the test designed for? What is the basis for
considering whether the test is applicable for the target group?
4
b. Content Validity:
(The relevance of the test content to the goal of the test)
Is the language skills constructed for the test relevant to the purpose of the test? To
what extent does the test task relate to purposes of the test? Does the test sufficiently
predict test takers’ desired performance? What characteristics of the test rubric (if any)
are likely to cause different test takers to perform differently?
c. Reliability: (The consistency of measurement)
Is the test free from errors such as ambiguity of test instructions, misleading samples,
etc? Are there any ambiguous items? Are the test scores likely to be consistent across
different testing situations?
d. Scoring Method and Score Reporting
What is the scoring method used: criterion-referenced or norm-referenced? How are
the test results reported to test-takers? Are they clear and consistent with the intended
use of the test? What materials and resources are available to aid in interpreting test
results?
e. Impact of the test
To what extent does the feedback that test takers receive from the test relate to their
actual language use? How relevant and appropriate are the test scores to the decisions
to be made? Is the purpose of the test consistent with instructional goals of teachers
and curriculum? What are the potential consequences, both positive and negative, for
society and educational system of using this test?
Total Points
4
4
4
4
20
The following are several examples of tests that you could choose to evaluate. You are also free to
choose other standardized tests on language that you would like to evaluate.
§
Test of English as A Foreign Language (TOEFL)
§
International Language Testing System (IELTS)
§
Michigan English Language Assessment Battery (MELAB)
§
Minnesota Language Proficiency Assessment
§
Woodcock-Munoz
§
Language Assessment Scales (LAS)
§
Idea Proficiency Test (IPT)
Guideline questions above are adapted from Test Evaluation by Lawrence M. Rudner, ERIC/AE 12/1993
and Language Testing in Practice by Lyle F. Bachman & Adrian S. Palmer