Cummins, R.A. & Gullone, E. (2000). Why we should not use 5-point Likert scales: The case
for subjective quality of life measurement. Proceedings, Second International Conference on
Quality of Life in Cities (pp.74-93). Singapore: National University of Singapore.
Why we should not use 5-point Likert scales: The case
for subjective quality of life measurement
Robert A. Cummins
School of Psychology
Deakin University
Eleonora Gullone
and
Department of Psychology
Monash University
Key words: Likert scale, history, reliability, sensitivity.
Correspondence:
Robert A. Cummins
Professor of Psychology
Deakin University
221 Burwood Highway
Melbourne, Victoria 3125
Australia
E-mail: robert.cummins@deakin.edu.au
Abstract
An argument is presented that the Likert scales commonly employed to measure subjective
quality of life (SQOL) are not sufficiently sensitive for the purpose of using SQOL as a
measure of outcome. A review of the literature indicates that expanding the number of
choice-points beyond 5- or 7-points does not systematically damage scale reliability, yet such
an increase does increase scale sensitivity. It is also argued that naming the Likert scale
categories detracts from the interval nature of the derived data. As a consequence it is
recommended that SQOL be measured using 10-point, end-defined scales.
Acknowledgement
I thank Betina Gardner for her assistance in the preparation of this document.
Subjective quality of life (SQOL) is gaining prominence as a measure of intervention
effectiveness. This is most evident in the fields of human service delivery and medicine
where a very large number of instruments have been devised to measure SQOL in some form.
Typically, each instrument will comprise 10 to 20 items, with each item scored on a 3- to 5point Likert scale.
While this recognition of personal, perceived well-being as a measure of outcome
effectiveness represents a major conceptual advance, it is time to take the forms of
instrumentation more seriously. Cummins (1997) lists around 400 instruments that have been
devised to measure SQOL or related constructs, and almost all rely on the use of ‘Likert-type’
scales. However, a glance at this collection reveals an apparent absence of rules by which
researchers have designed their scales. The number of choice points can vary from two to
100, some use unidimensional scales (e.g. from ‘no satisfaction’ to ‘complete satisfaction),
some use bidimensional scales (e.g. from ‘complete dissatisfaction’ to ‘complete
satisfaction’), some use a neutral scale mid-point while others do not, some use extreme
anchors (e.g. ‘Terrible’) while others use mild anchors (e.g. ‘Dissatisfied’), and so on. Each
of these different forms is known to influence the response pattern that people make to Likert
scales, and yet there has been almost no attention to such issues in relation to SQOL
measurement.
At one level it could be argued this is relatively unimportant. The Likert scale has a fairly
robust character which has proved to be reliable over a wide variety of forms, and the larger
issues concern such matters as the item content of scales related to their construct validity.
While such concerns are valid there is one aspect of Likert scale construction which is at least
of equal importance when the data are used as measures of outcome. As suggested by Guyatt
and Jaeschke (1990), this is the issue of measurement sensitivity, and it is really quite curious
that this crucial parameter has been virtually ignored.
The typical Likert scale offers 5- or 7-choice points which, of itself, is hardly likely to exploit
the discriminative capacity of most people in terms of their perceived well-being. But even
this modest array of response choices is actually reduced by a number of factors, some of
which are as follows:
1. SQOL is not free to vary over its entire range subject to the contingency of personal
circumstance. It is constrained in range by its trait characteristic (see for example
McCauley & Bremer, 1991) which causes it to behave as though it is held under
homeostatic control. For example, the population mean for life satisfaction, a common
measure of SQOL, is held within the normative range of 70-80 percentage of the scale
maximum within Western populations (Cummins, 1995, 1998).
Consequently,
respondents will normally experience SQOL variations that are limited to only a
proportion of the available scale.
2. As noted above, SQOL data display a natural negative skew when measured using
ordinary unipolar or bidimensional Likert scales (Watson, 1930; Cummins, 1997).
Consequently, again the majority of respondents are employing only the higher or
positive half of the scale in order to register their judgement.
3. Responses to Likert scales comprise a high degree of response-set. For example,
Peabody (1962) found an average correlation of .54±.24 between individuals’ intensity of
agreement or disagreement to sets of attitudinal questions. Moreover, he calculated the
relative contribution of response direction (positive or negative around a neutral midpoint) and response intensity (response distance from the neutral mid-point to an end of
the scale) to a composite score, and found that intensity contributed only 10 percent to the
composite variation. This has important implications for the detection of differences
between QOL domains, such as provided by the Comprehensive Quality of Life Scale
(Cummins, 1997). Maximum scale sensitivity is required in order to measure such
differential levels of intensity.
4. Vokman (1951) first drew attention to the ‘variable series effect’ (see also Upshaw,
1962). Here it is proposed that the stimulus range is a major determinate of the value
assigned to items in a series. Essentially, if people are presented with a scale in which
their attitude is non-mid-point, then they will subjectively divide the range between the
values that they recognize as being consistent with their own attitude range and, as a
consequence, exhibit a narrower response range than the presented scale intends.
So the conclusion to be drawn is that Likert scales, as currently constructed, are likely to
represent very blunt instruments by which to judge change in SQOL. In order to see why this
has come about it is necessary to outline some of the history of scale development.
A history of Scale Construction
Likert (1932) was not the first to obtain subjective ratings on a printed scale, and it is most
interesting to note that the early scale developers used far more sensitive scales than we
currently employ. Freyd (1923) discusses the various forms of scale available at that time
and notes that they tended to be based on 10-point or 100-point formats. For example, the
‘Decile’ scale comprises a number of statements corresponding to different levels of
construct ‘strength’ against numbers from 0 – 10. This numbering system is undoubtedly the
most intuitive and easy to conceptualize. The traditional counting task for children involves
their fingers or toes. It also has the advantage of having a perception of equal psychometric
distance between the scale points. This is an essential supposition when such scale are used
in combinations with parametric statistics, even though this condition is known to be violated
using Likert’s scale to varying degrees (see e.g. Ferguson, 1941).
Freyd then introduced his ‘Graphic rating method’ which had the following form:
[item] Does he appear neat or slovenly in his dress?
Extremely neat
and clean. Almost
a dude.
Appropriately and
neatly dressed.
Inconspicuous
in dress.
Somewhat
careless in his
dress.
Very slovenly
and unkempt.
The above scale was intended to be used in conjunction with job interviews, and raters were
instructed as follows:
“When you have satisfied yourself on the standing of this person in the trait on which
you are rating him, place a check at the appropriate point on the horizontal line. You
do not have to place your check directly above a descriptive phrase. You may place
your check at any point on the line.” (p.88).
He then recommended scoring the responses by dividing the line into 10 or 20 equal
intervals.
A few years later, Watson (1930) published a similar scale to measure an aspect of SQOL as
follows:
Most miserable
of all
About three-fourths
of the population
are happier than you
are
The average
person of your
own age and sex
Happier, on the
whole, than threefourths of the
population of similar
age and sex
Happiest
of all
Instructions to respondents were:
“Comparing yourself with other persons of the same age and sex how do you feel you
should rate your own general happiness? Place a short vertical mark across the line
below to indicate about where you belong. Consider your average state over several
months.” (p.84)
The scale was then scored 0-100.
Then, in 1932, Likert produced his scale which had the following form:
Strongly
Approve
Approve
Undecided
Disapprove
Strongly
Disapprove
While this format is clearly derivative from the previous examples, it importantly and
drastically reduces the number of effective choice-points in two ways. First, the scoring
system is no longer continuous. Respondents are now required to mark the line only adjacent
to one of the labels, making the scoring system 1-5. Second, he has introduced the
bidimensional scale with a neutral mid-point.
More than six decades have passed since Likert’s formulation was published and it
instructive to consider why this original form has remained so popular. The reasons include
the type of psychometric investigation to which it has been subjected, the difficulty of
generating substantially larger numbers of labeled choice points, and the complex nature of
alternative scales. Each of these issues will now be considered with the aim of demonstrating
that the 5-point format has survived because it has utility for some types of measurement, but
that these have not included the issue of scale sensitivity in the measurement of SQOL.
Psychometric properties of the Likert scale
The three basic properties of Likert scales are reliability, validity, and sensitivity. And the
extent to which psychometric research has concentrated on the former is astonishing. Almost
40 years after the scale had been published, Jacoby and Matell (1971) stated “ … most of the
psychometric literature, dealing with the number-of-alternatives problem emphasizes
reliability as the major criterion in the number of scale points. However, the ultimate
criterion is the effect a change in the number of scale points has on the validity of the scale.
An intensive literature search failed to reveal any empirical investigation addressed to this
question.” (p.496).
While the situation has changed somewhat over the intervening period it is still true that the
kinds of psychometric data researchers report in support of their scales are generally
restricted to reliability (internal and test-retest) and convergent/divergent validity which is,
itself, highly dependent on reliability. This narrow focus has led, inexorably, to the view that
smaller, rather than larger number of scale points are advantageous to measurement (e.g.
Cronbach, 1946). The reason for this has been twofold. First, early research showed that
increased numbers of choice-points means that there is more scope for people to display
response-sets (see Cronbach, 1950, for a review), and this scope is minimized in
dichotomously scored scales. Second, later research produced considerable divergence of
opinion on the merit of shorter vs. longer scales. Given such ambiguity, the most pragmatic
choice favors shorter scales since they reduce response time in large surveys where they are
most commonly employed.
But now it is time to revisit this literature from a different perspective. Not whether evidence
can be found to support the use of simple-choice Likert scales in surveys, but whether
evidence can be found to support an expanded number of choice points for the purpose of
SQOL measurement. In terms of this appraisal there are two pertinent issues. First whether
such expansion is likely to enhance measurement sensitivity, and second whether such
expansion is detrimental to the other psychometric characteristics of scale reliability and
validity.
When Cronbach (1946, 1950) railed at the use of multiple-point scales he did so largely in the
context of educational tests where response-sets were evident in knowledge-based questions,
most particularly where the student did not know the answer and so engaged in such
response-sets as acquiescence, by favoring ‘true’ response modes over false. But as
Cronbach acknowledged “If a situation is structured for the student, so that he knows the
answer required, he responds directly to the content of the item and response sets probably
are unimportant.” (1946, p.483). Surely this, then, places the SQOL scales in a quite
different context. The SQOL questions are simple, respondents have a view that they can
express on the scale provided, and the questions are not knowledge based. And, indeed, the
reliability data on non-knowledge based items do seem to indicate that response-sets are
generally ‘unimportant’ as will be demonstrated.
In terms of sensitivity the issue seems intuitive. Few people would feel their discriminative
capacity for perceived well-being to be limited to five levels of experience. Moreover, as
indicated earlier, there are a number of scale-construction factors which tend to reduce the
effective choice still further. So, it would be expected that the empirical literature would
support this view, and it does. Indeed, opinion seems to be unanimous; increasing the
number of scale points increases scale sensitivity. Thus, for example, Diefenbach et al.
(1993) found a 7-point scale to be more sensitive than a 5-point scale while Russell and
Bobko (1992) found that data from a 15-point scale increased regression analysis effect sizes
by 93 percent over those from a 5-point scale. As noted by Jaeschke and Guyatt (1990) in the
context of medical QOL scales, it seems clear that 5-point scales, at the least, do not provide
sufficient sensitivity to detect small, clinically significant differences. So what, then, are the
psychometric impediments to using expanded Likert scales?
The main issue here is scale reliability, and it is true that some authors (e.g. Bardo & Yeager,
1982a&b; Bardo, et al., 1982) have concluded that scale reliability decreases as the number
of choice-points exceeds two. However, these authors had the specific intention of detecting
response-sets with a methodology that involved asking people to respond randomly to
presented scales, an approach which has little relevance for SQOL measurement.
Another source of data generally favoring simple scales has been derived from computer
simulations. For example, Lissitz and Green (1975) determined that Likert-scale reliability
increased from 2- to 5-points, but that no further gains were made beyond this point.
However, later and larger-scale simulations (Cicchetti, et al., 1985; Jenkins & Taber, 1977)
extended this upward range. Using monte carlo methodology these authors found that while
the largest changes occurred over the range 2-points to 5/7-points, gradual increases beyond
5/7-points were evident in a range of psychometric parameters including inter-rater reliability
and judgement accuracy. Such findings imply no impediment to the generation of more
complex scales from consideration of reliability.
The conclusions reached by researchers using human-generated data have been more
variable. On the minimalist side, Peabody (1962) argued the case that agree/disagree scales
should simply be scored dichotomously according to the direction of response. This was
based on his finding that, when people responded to attitude scales, a ‘composite score’
derived by combining direction of response (agree or disagree) with the intensity of response
(extent of agreement or disagreement) was dominated by the former. In fact, only 10 percent
of composite score variation could be attributed to intensity, as opposed to the 70-80 percent
attributed to direction. He also found a high degree of response-set within the intensity
dimension. While all of this led him to recommend dichotomous scales two factors should be
noted. First, he used a 6-point scale and acknowledged that with an increased number of
choice-points to 9 or 11 “the reasoning used with regard to six-point scales would imply that
the extremeness component might play a more significant role.” (p.72). Second, his data
formed near-normal distributions, as opposed to the determined negative skew of SQOL data.
In relation to this he acknowledged that a similar calculation would be unreliable since
subjects tend to respond in the same direction to nearly all items.
Other studies which have argued for fewer numbers of choice points are McKelvie (1978)
and Chang (1994). The former compared 5-, 7-, and 11-point scales over two studies, and
two rating tasks. In the main no differences were found between the scales on inter-rater
reliability (agreement), test-retest reliability, and validity (a tone judgement task). Where
effects were found on reliability, they tended to be contradictory, showing quite different
trends for raw and transformed data. In terms of validity, the 5-point scale was clearly
inferior. Curiously, however, the author concluded the 5-point scale to be superior, despite
the weight of empirical evidence to the contrary.
Chang (1994) took a more sophisticated approach. Using multitrait-multimethod to separate
trait and method variance he found lower internal reliability within a 6-point agree/disagree
scale than within a 4-point scale, but no difference between the scales in terms of criterionrelated reliability. The author concludes that increasing the number of scale points creates
opportunities for response sets to arise. However, he also acknowledges “The issue of
selecting 4- versus 6-point scales may not be generally resolvable, but may rather depend on
the empirical setting.” (p.205).
In summary for the negative, none of these authors have presented a strong generalist case
against the use of more complex scales in terms of reliability, and certainly not in the context
of negatively skewed data. On the other hand, other researchers have found no change in
reliability over scales ranging from 2 to 19 points (Test-retest and internal, Matell & Jacoby,
1971) or even 5 to 100 points (Diefenbach, et al., 1993). Still others have found increased
rater reliability (the ability of single raters to discriminate differences between the rated
stimuli:Bendig, 1954) and test reliability (the consistency of individual differences in the
assignment of high and low ratings) when employing from two to five categories (Bendig,
1954), and an increased internal reliability from two- to six-points (Komorita & Graham,
1963). Finally, in a meta-analysis of 131 studies in the marketing research literature,
Churchill and Peter (1984) found a positive relationship between internal reliability and the
number of scale choice points (mean number of points 5.8±2.3, range 1-20). Their regression
analysis revealed that the number of choice points explained five percent of the reliability
variance.
From all of these data it may be reasonably concluded that increasing the response options
beyond 7-points does not systematically detract from scale reliability, a conclusion shared by
others (e.g. Russell & Bobko, 1992). So, since many people will have a discriminative
capacity that exceeds 7 points, restricting people to such scales results in a loss of potentially
discriminative data. This is most particularly relevant in relation to SQOL measurement
where the data are skewed. However, there are some potential difficulties in generating
expanded scales, and one of these involves the tradition of using category names.
The issue of categorical naming
Likert (1932) named all of his five categories and the received wisdom is that this is a good
idea. Andrews and Withey (1976), authors of the most widely cited document in the QOL
literature, endorse category naming because, they suggest, it enhances comparability in the
way respondents use the scales, and it makes it possible to know exactly what the respondent
was endorsing. However they also note that it is not really true. The use of such terms as
‘good’ does not, in fact, ensure a standardized point of reference. For example, a ‘good’
house has very different meanings depending on a respondent’s socio-economic status, and
high ambiguity in the allocation of values to scale categories has been documented for some
time (e.g. Jones & Thurstone, 1955). So these reasons do not withstand scrutiny. Not only
that, but there are other, powerful reasons to actually avoid category naming and these ideas
have not been given the publicity they deserve.
The Likert scale makes the assumption that the psychometric distance between categories is
equal. This aspect of scale construction is always dutifully portrayed as an equally-spaced
visual image, comprising marks on a horizontal line or a series of boxes. It may even be
reinforced by a linear numbering system, for example from 1 to 7, with each successive
integer corresponding to the next printed category. And then we add category names.
The clear implication is that these categorical names exhibit the same internal scaling as the
printed scale and numbers suggest. This, however, is wrong, sometimes very wrong, and the
data demonstrating this have been available for a considerable period of time (Cronbach,
1946).
Table 1
Estimates of frequency (percent)
Category
Never
Rarely
Very unusual
Unusual
Seldom
Very unlikely
Once or twice
Infrequently
Unlikely
Once in a while
A
6.7
B
STUDY
C
D
0.2*±.6
5.3±3.6
Maximum
Difference
1.4
9.0
22.0
25.6
28.4±11.3
28.9
31.1
31.4±10.4
32.2
Now and then
Sometimes
Occasionally
Pretty often
Often
47.8
50.0
55.6
56.7
60.0
Frequently
Repeatedly
Usually
Very probable
Almost always
65.6
72.2
Most of the time
Always
95.6
33.4±17.0
19.5±15.1
35.0
33.0
16.6
36.1
60.9±13.1
58.5
2.4
81.2±10.4
75.9±9.8
30.0
45.9
82.5±8.8
94.2±4.9
99.5*±2.0
83.1±8.8
16.4
* Participants were told that always = 100% and never = 0%
Table code:
A:
B:
C:
D:
Spector (1976) from College students.
Kenney (1981) from ‘professionals’. Respondents were told that always = 100%, and never
= 0%.
Nakao & Axelrod (1983) from ‘non-physicians’.
Tavana et al. (1997) from financial strategy experts.
The data in Table 1 indicate the percentage allocated to category labels of ‘frequency’ by
respondents in four separate studies. The following observations can be made:
1.
With two exceptions (‘rarely’ and ‘often’) there is considerable discrepancy between the
average category values derived from the studies. The worst labels in this regard are
‘occasionally’ and ‘usually’ which differ by 36.1 percent and 45.9 percent respectively.
2.
The relative ordering of terms is not consistent between studies. See, for example,
‘sometimes’ in relation to ‘occasionally’. The relative ordering of terms within studies,
however, is always consistent with the presented Likert scale order.
3.
Study B (Kenney, 1981) demonstrates a clear pattern of increasing intra-category
variance towards the middle categories. But this is likely to be an artifact of their
methodology where participants were told that ‘never = 0%’ and ‘always = 100%’. In
the absence of such cuing, Jones and Thurstone (1955) found exactly the opposite result,
that the standard deviation of score allocation on a 9-point scale (+4 to –4) increased
from the more neutral, central categories (e.g. good) to the extreme categories (e.g. Best
of all). Actually, the highest variance was accorded to extreme negative values (e.g.
Loath, despise), but perhaps this is because they are unusual words and their relative
meaning was uncertain to some respondents.
The conclusion from all of this is obvious. People have such a varying interpretation of the
numerical value accorded to frequency category labels, that their application in Likert scales
is simply detracting form the interval-nature of such scales. The reason that the intra-study
category values always accord with the presented order is that respondents are being cued by
the relative position of category labels on the printed scale. Support for this has been
provided by Solomon and Kopelman (1984) who found higher scale interval reliability when
items were grouped by content and labeled, rather than being randomly distributed.
It is also notable that the category labels of ‘frequency’ which form Table I, should provide at
least some response cues to their numerical equivalence. This type of cue is far less evident
in the category labels used in most SQOL scales, and so the latter are probably even less
consistently employed than the frequency data. Some data are available to support this.
Ware and Gander (1994) used the Thurstone method of equal-appearing intervals to calculate
the following distances between category labels used in the SF-36 (Ware & Sherbourne,
1992) as follows: Poor (1.0), Fair (2.3), Good (3.4), Very Good (4.3), and Excellent (5.0). It
can be seen that the distance between the lowest two categories (1.3) is about double that
between the highest two categories (0.7). A comparison can also be made between the
ratings of these same categories by Spector (1976). In equivalent units he found the
separation of ‘poor’ and ‘fair’ to be 2.3, while that between ‘good’ and ‘excellent’ to be 1.1
units. It can be seen that the disparities between low and high adjacent category labels is in
the same direction but of even greater magnitude.
It can also be noted that the business of naming Likert categories constitutes a severe
impediment to the expansion of scales due to the difficulty of finding appropriate categorical
names. Consider, for example, the 9-point scale generated by Roy Morgan Research (1993)
as: Delighted, very pleased, pleased, mostly satisfied, mixed feelings, mostly dissatisfied,
unhappy, very unhappy, terrible. This scale assumes that the psychometric distance from
neutrality to ‘pleased’ and to ‘unhappy’ are equivalent. There are no data known to me that
support this view and yet such assumptions are forced by the inability of our language to
make fine discriminations between affective states. So it can be concluded that the addition
of category names to Likert scales not only detracts from the interval nature of the scale but
also makes it difficult to generate expanded choice formats. Therefore since expanded Likert
scales are desirable for SQOL measurement, as has been previously argued, the appropriate
scale format may be a 10-point, end-defined scale.
Justification for a 10-point, end-defined scale
The use of end-defined scales was pioneered by Jones and Thurstone (1955). They generated
a 9-point scale in relation to food preference, anchored by ‘greatest like/dislike’ (+4/-4), with
a central category of ‘Neither like nor dislike’ (0), and with the intermediate categories
labeled only by their appropriate integer. So, do such scales produce data that is different
from conventionally labeled Likert scales?
The answer appears to be in the negative. Matell and Jacoby (1972) used verbally anchored
adjective statements related to civic beliefs, with the number of intervening points varying
from two to 19. Apart from the fact that testing time increased with >12-point formats, no
differences were found on the proportion of scale utilized (>3-points) or in the proportion of
‘uncertain’ responses (>5-points). Similarly, Wyatt and Meyers (1987) using a 5-point scale,
and Dixon et al. (1984) using a 6-point scale, found no systematic differences between the
data from end-defined and conventional Likert scales. From this it can be concluded that the
end-defined format seems not to bias the data in any particular way. It is also interesting to
note that an increasing number of recent authors (e.g. Hooker & Siegler, 1993; Watkins, et
al., 1998) are using 10-point end-defined scales.
Conclusion
Likert scales in common use, within the field of SQOL measurement, were devised for a
quite different context. Of particular importance in this regard is the fact that SQOL data are
negatively skewed, which means that most people will respond only to a restricted portion of
the conventional scale. Moreover, when SQOL is used as a measure of outcome, scale
sensitivity becomes a critical concern since this construct has a high trait component, and
small deviations are highly meaningful. So, it is proposed, the number of choice points needs
to be expanded.
Such expansion appears not to systematically influence scale reliability, and is therefore
psychometrically feasible, but is made difficult by the convention of naming all response
categories. It has been argued that this naming is quite unnecessary and actually detracts
from the interval nature of the scale. So the solution proposed is to adopt 10-point, enddefined scales. These offer a form of rating (one to ten) which lies within common
experience and produce increased sensitivity of the measurement instrument.
References
Andrews, F.M., & Withey, S.B. (1976). Social Indicators of well-being: Americans’
perceptions of life quality. New York: Plenum Press.
Bardo, J.W., & Yeager, S.J. (1982a). Consistency of response style across types of response
formats. Perceptual and Motor Skills, 55, 307-310.
Bardo, J.W., & Yeager, S.J. (1982b). Note on reliability of fixed-response formats.
Perceptual and Motor Skills, 54, 1163-1166.
Bardo, J.W., Yeager, S.J., & Klingsporn, M.J. (1982). Preliminary assessment of formatspecific control tendency and leniency error in summated rating scales. Perceptual
and Motor Skills, 54, 227-234.
Bendig, A.W. (1954). Reliability of short rating scales and the heterogeneity of the rated
stimuli. Journal of Applied Psychology , 38, 167-170.
Chang, L. (1994). A psychometric evaluation of 4-point and 6-point Likert-type scales in
relation to reliability and validity. Applied Psychological Measurement, 18, 205-216.
Cicchetti, D.V., Showalter, D., & Tyrer, P.J. (1985). The effect of number of rating scale
categories on levels of interater reliability: A Monte Carlo investigation. Applied
Psychological Measurement, 9, 31-36.
Cronbach, L.J. (1946). Response sets and test validity. Educational and Psychological
Measurement, 6, 475-494.
Cronbach, L.J. (1950). Further evidence on response sets and test design. Educational and
Psychological Measurement, 10, 3-31.
Cummins, R.A. (1995). On the trail of the gold standard for subjective well-being. Social
Indicators Research, 35, 179-200.
Cummins, R.A. (1997). The Directory of Instruments to measure quality of life and cognate
areas of study. Fourth Edition. Melbourne: Deakin University.
Cummins, R.A. (1997). The Comprehensive Quality of Life Scale – Adult. Fifth Edition.
Melbourne: Deakin University.
Cummins, R.A. (1998). The second approximation to an international standard for life
satisfaction. Social Indicators Research, 43, 307-334.
Diefenbach, M.A., Weinstein, N.D., & O’Reilly, J. (1993). Scales for assessing perceptions
of health hazard susceptibility. Health Education Research, 8, 181-192.
Dixon, P.N., Bobo, M., & Stevick, R.A. (1984). Response differences and preferences for
all-category-defined and end-defined Likert formats. Educational and Psychological
Measurement, 44, 61-66.
Ferguson, L.W. (1941). A study of the Likert technique of attitude scale construction.
Journal of Social Psychology, 13, 51-57.
Freyd, M. (1923). The graphic rating scale. Journal of Educational Psychology, 14, 83-102.
Guyatt, G.H., & Jaeschker, R. (1990). Measurement in clinical trials: Choosing the
appropriate approach. In: B. Spilker (Ed.), Quality of life assessment in clinical trials
(pp.37-46). New York: Raven Press.
Hooker, K., & Siegler, I.C. (1993). Life goals, satisfaction, and self-rated health: Preliminary
findings. Experimental aging research, 19, 97-110.
Jacoby, J., & Matell, M.S. (1971). Three-point Likert scales are good enough. Journal of
Marketing Research, 8, 495-500.
Jaeschke, R., & Guyatt, G.H. (1990). How to develop and validate a new quality of life
instrument. In: B. Spilker (Ed.) Quality of life assessment in clinical trials (pp.4757). New York: Raven Press.
Jenkins, G.D., & Taber, T.D. (1977). A monte carlo study of factors affecting three indices
of composite scale reliability. Journal of Applied Psychology, 62, 392-398.
Jones, L.V., & Thurstone, L.L. (1955). The psychophysics of semantics. Journal of Applied
Psychology, 39, 31-36.
Kenney, R.M. (1981). Between never and always. New England Journal of Medicine, 305,
1097-8.
Komorita, S.S., & Graham, W.K. (1965). Number of scale points and the reliability of scales.
Educational and Psychological Measurement, 25, 987-995.
Likert, R. (1932). A technique for the measurement of attitudes. Archives in Psychology,
140, 1-55.
Lissitz, R.W., & Green, S.B. (1975). Effect of the number of scale points on reliability: A
Monte Carlo approach. Journal of Applied Psychology, 60, 10-13.
Matell, M.S., & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert
scale items? Study 1: Reliability and validity. Educational and Psychological
Measurement, 31, 657-674.
McCauley, C., & Bremer, B.A. (1991). Subjective quality of life for evaluating medical
intervention. Evaluation and the Health Professions, 14, 371-387.
McKelvie, S.J. (1978). Graphic rating scale – How many categories? British Journal of
Psychology, 69, 185-202.
Nakao, M.A., & Axelrod, S. (1983). Numbers are better than words: Verbal specifications of
frequency have no place in medicine. American Journal of Medicine, 74, 1061-1065.
Peabody, D. (1962). Two components in bipolar scales: Direction and extremeness.
Psychological Review, 69, 65-73.
Roy Morgan Research (1993). International values audit, 22/23 May. Melbourne: Roy
Morgan Research Centre.
Russell, C., & Bobko, P. (1992). Moderated regression analysis and Likert scales: Too
coarse for comfort. Journal of Applied Psychology, 77, 336-342.
Solomon, E., & Kopelman, R.E. (1984). Questionnaire format and scale reliability: An
examination of three modes of item presentation. Psychological Reports, 54, 447452.
Spector, P.E. (1976). Choosing response categories for summated rating scales. Journal of
Applied Psychology, 61, 374-375.
Tavana, M., Kennedy, D.T., & Mohebbi, B. (1997). An applied study using the analytic
hierarchy process to translate common verbal phrases to numerical probabilities.
Journal of Behavioral Decision Making, 10, 133-150.
Upshaw, H.S. (1962). Own attitude as an anchor in equal-appearing intervals. Journal of
Abnormal and Social Psychology, 64, 85-96.
Vokman, P. (1951). In: Upshaw, H.S. (1962). Own attitude as an anchor in equal-appearing
intervals. Journal of Abnormal and Social Psychology, 64, 85-96.
Ware, J.E., & Gander, B. (1994). The SF-36 health survey: Development and use in mental
health research and the IQOLA project. International Journal of Mental Health, 23,
49-73.
Ware, J.E., & Sherbourne, C.D. (1992). The MOS 36-item short-form health survey (SF36). Medical Care, 30, 473-481.
Watkins, D., Akande, A., Fleming, J., et al. (1998). Cultural dimensions, gender, and the
nature of self-concept: A fourteen-country study. International Journal of
Psychology, 33, 17-31.
Watson, G.B. (1930). Happiness among adult students of education. Journal of Educational
Psychology, 21, 79-109.
Wyatt, R.C., & Meyers, L.S. (1987). Psychometric properties of four 5-point Likert-type
response scales. Educational and Psychological Measurement, 47, 27-35.