Matakuliah
Tahun
: <<Kode>>/<<Nama mtkul>>
: <<Tahun Pembuatan>>
BASIC CONSIDERATIONS in Test Design
2
Pertemuan 16
1
RELIABILITY
CONCEPT of RELIABILITY :
1.
2.
3.
4.
The Reliability Coefficient
The Standard Error of measurement and the true score
Scorer Reliability
How to make Tests more reliable.
2
RELIABILITY
It concerns with how far we can depend on
the results that a test produces or in other
words, could the results be produced
consistently.
3
The Reliability coefficients:
The ideal reliability coefficient is 1.
A test with a reliability coefficient of 1 is one which would give
precisely the same results for a particular set of candidates
regardless of when it happened to be administered.
A test with a reliability coefficient of zero would give sets of results
quite unconnected with each other.
Lado says that good vocabulary, structure and reading tests are
usually in the range of .90 to .99 , while auditory comprehension
tests are often in the .80 to .89 range. Oral production tests may be
in the .70 to .79 range.
4
How to arrive at the reliability coefficient ?
The requirement is to have two sets of scores for comparison, by:
1. getting a group of subjects to take the same test twice (test-retest
method);
2. using two different forms of the same test (alternate forms
method).
5
The standard error of measurement and
the true score
While the reliability coefficient allows us to compare the reliability of
tests, it does not tell us directly how close an individual’s actual
score is to what he or she might have scored on another occasion.
With a little further calculation, however, it is possible to estimate
how close a person’s actual score is to what is called their “true
score’.
For the calculation, see appendix 1, “Testing for Language
Teachers”, Arthur Hughes, page 159
6
HOW TO MAKE TESTS MORE RELIABLE
1. take enough samples of behaviour
2. do not allow candidates too much freedom
3. write unambiguous items
4. provide clear and explicit instructions
5. ensure that tests are well laid out and perfectly legible
6. candidates should be familiar with format and testing techniques
7. provide uniform and non-distracting conditions of administration
7
8. use items that permit scoring which is as objective as possible
9. make comparison between candidates as direct as possible
10. provide a detailed scoring key
11. train scorers
12. agree acceptable responses and appropriate scores at outset of
scoring
13. identify candidates by number, not name
14. employ multiple, independent scoring.
8