Reliability
Presentation
Test-Retest
James Blackwood – AED 615
Fall Semester 2006
Test-Retest Reliability




Test-Retest method of determining
reliability is accomplished by administering
a test to a group
After a period of time has passed, the same
test is re-administered to the same group.
It is also known as stability reliability
It is used in both qualitative and
quantitative research (qualitative requires a
different technique of analysis)
Reliability Coefficient
1.
2.
3.
After two tests have been administered, a reliability
coefficient is calculated to determine the
relationship between the two scores obtained.
If the same results are obtained from the two tests
then the coefficient = 1.0
The coefficient is influenced by the amount of time
that has passed between the administration of the
two tests.
Reliability Coefficient Measurement
1.
2.
The reliability coefficient is expected to be
lower the longer the time interval between
the tests due to the possibility of changes in
the population taking the test.
The shorter the time gap, the higher the
correlation; the longer the time gap, the
lower the correlation.
Test-Retest Issues



There cannot be any measurable change in
the construct being measured between the
two tests.
This method will not work when measuring a
variable that is not stable in an individual.
Unless the instrument is reliable,
relationships with other variables in the study
will not be identified.
Testing..



You can obtain considerably different
estimates of reliability depending on the
interval between tests.
For educational research, examination of
scores over a two to three month period is
sufficient for test-retest reliability verification.
The time interval between the two tests should
always be reported when using test-retest as a
measure of reliability.
Test-Retest Equation
Test-Retest Issues
1.
2.
3.
4.
5.
Requires twice the data collection
Population for the test would need to be
willing to repeat the test (impractical)
Higher costs due to multiple tests being
administered
Only works well when practical (better for
smaller population rather than a large one)
Researcher may not be able to retest the
population
Examples of Test-Retest




Educational assessment
Drug testing
Testing Measurement Equipment
Medical Evaluations
Research Literature Example
American Journal of Agricultural Economics
Volume 71 - Number 1 (Feb 1989), pp 76-84
Test-Retest Reliability of the Contingent
Valuation Method: A Comparison of General
Population and Visitor Responses
John B. Loomis
Abstract:
The reliability of the contingent valuation method is evaluated by resurveying the same general households and
visitors nine months after their original survey. Test-retest correlations on willingness to pay are statistically
significant and ranged from .422 for the general population sample to .782 for the visitor sample. Using a
paired T-test, there was no statistical difference between an individual's first and second reported willingness
to pay. Chow tests comparing the original and resurvey willingness-to-pay functions showed no statistical
difference at the .01 level. Reported willingness to pay is reasonably stable over the time period surveyed.
References
Fraenkel, J. R., & Wallen, N. E. (2006). How to design and evaluate research in
education (6th ed.). New York: Mc-Graw-Hill.
Guttman, L. (1946). The test-retest reliability of qualitative data. Psychometrika, 11(2),
81-95. Abstract retrieved October 15, 2006 from Springer Link database.
Loomis, J. B. (1989) Test-retest reliability of the contingent valuation method: A
comparison of general population and visitor responses. American Journal of Agricultural
Economics, 71(1), 78-84.
Trochim, W. M., (2006). Types of reliability. Research Methods Knowledge Base. Retrieved from
http://www.socialresearchmethods.net/kb/reltypes.htm