Meaning
Definition
Types
Factors influencing
Improve of test
Reliability is one of the important characteristics of
any test.
It refer to the precision or accuracy of the
measurement of score.
Reliability refers to the stability of a test measure or
protocol.
Rosenthal(1991): Reliability is a major concern when
a psychological test is used to measure some attribute
or behaviour.
Anastasi (1968): Reliability refers to the consistency
of scores obtained by the same individuals when reexamined with test on different occasions, or with
different sets of equivalent items, or under other
variable examining conditions.
◦ Measurement experts and many educators believe that every
measurement device should possess certain qualities.
◦ The two most common technical concepts in measurement are
reliability and validity.
◦ Reliability----------------CONSISTENCY
◦ Validity--------------------ACCURACY
◦ The degree of consistency between two measures of the same
thing.
◦ The measure of how stable, dependable, trustworthy, and
consistent a test is in measuring the same thing each time.
Keys of reliability assessment:
◦ Stability: related to time consistency
◦ Internal: related to the instruments
◦ Interrater: related to the examiners criterion
Reliability Value:
values for reliability coefficients range from 0 to 1.0. A coefficient of 0
means no reliability and 1.0 means perfect reliability. Since all tests have
some error, reliability coefficients never reach 1.0. Generally, if the
reliability of a standardized test is above .80, it is said to have very good
reliability; if it is below .50, it would not be considered a very reliable test.
Three important types
◦ Test –Retest Reliability
◦ Split – half or Internal Consistency Reliability
◦ Parallel forms reliability or Equivalent - forms
In test –retest reliability the single form of the test is
administered twice on the same sample with a
reasonable time gap.
In this way two administration of the same form of
the two independent sets of scores.
The two sets, when correlated, give the value of the
reliability coefficient.
Same Test
Same Sample
Different Times
Measure instrument at two times for multiple
persons.
Compute correlation between the two measures.
Assumes there is no change in the underlying trait
between time 1 and time 2.
Contributing factors:
◦ (1) Clear instructions for administrators, research
participants, and raters
◦ (2) Tasks/questions in participants’ first language or
target language at appropriate level of difficulty
◦ (3) Unambiguously phrased tasks/questions
Other name Internal Consistency reliability
It indicates the homogeneity of the test
This method the test is divided into two equal or
nearly halves
Common way of this test is the odd-even method
Indicates that subjects’ scores on some trials
consistently match their scores on other trials
Contributing factors:
◦ (1) Careful item writing, guided by item specifications
◦ (2) Field test and item analysis
◦ (3) Construction of tests with reference to item performance
This reliability various names such as,
◦ Alternative- forms reliability
◦ Equivalent –forms reliability
◦ Comparable – forms reliability
The alternative forms technique to estimate reliability
is similar to the test retest method, except that
different measures of a behaviour (rather than the
same measure) are collected at different times.
If the correlation between the alternative forms is
low, it could indicate that considerable measurement
error is present, because two different scales were
used.
Contributing factors:
◦ (1) The development of equivalent forms from specifications
that describe tool content
◦ (2) Trial of tools before data collection to ensure
equivalence
Group Variability
Guessing by the examinees
Environmental Conditions
When the group of examinees being tested is
homogenous in ability, the reliability of the test scores
is likely to be lowered.
But when the examinees vary widely in their range of
ability, that is, the group of examinees is a
heterogeneous one, the reliability of the test scores is
likely to be high.
Guessing in a test is an important source of
unreliability.
In two alternative response options there is a 50 %
chance of answering the items correctly on the basis of
the guess.
Testing environment should be uniform
Arrangement should be such that light, sound, and
other comforts are equal and uniform to all the
examinees.
Otherwise it will tend to lower the reliability of the test
scores
The group of examinees should be heterogeneous, that
is, the examinees should vary widely in their ability or
trait being measured.
Items should be homogenous.
Test should preferably be a longer one.
Items should be discriminatory ones.
C.Thatchinamoorhy (2015-16) Class Notes, AEX 812Advances in designs and techniques in Social Science
Research (2+1)
Dr.T.Rathakrishnan (2009). Advances & Challenges in
Agricultural Extension & Rural Development.
Rosenthal, R. and Rosnow, R. L. (1991). Essentials of
Behavioral Research: Methods and Data Analysis. Second
Edition. McGraw-Hill Publishing Company, pp. 46-65
http://www.socialresearchmethods.net/kb/mtmmmat.htm