Misresponse to Reversed and
Negated Items in Surveys: A Review
Bert Weijters
Hans Baumgartner
Misresponse to Reversed and Negated Items
Motivation
 Should reversed items be included in multi-item
summative scales?
□
The use of reversed items in surveys often leads to
problems (low reliabilities, poor fit, distorted factor structure);
□ Reversed items control for acquiescence, serve as cognitive
“speed bumps” and may encourage more complete
coverage of the construct’s domain of content;
 If reversed items are to be used, does it matter
whether the reversal is achieved through negation or
through other means?
 What’s the link between reversal and negation, what
types of MR result, what psychological mechanisms
are involved, and how can MR be avoided?
Misresponse to Reversed and Negated Items
Item reversal vs. item negation
 Authors often fail to draw a clear distinction between
reversals and negations and use ambiguous terms such
as ‘negatively worded items’, which makes it unclear
whether they refer to reversed or negated items, or both;
 Examples from the Material Values scale (Richins and
Dawson 1992):
□
□
□
It sometimes bothers me quite a bit that I can't afford to buy all
the things I’d like.
I have all the things I really need to enjoy life.
I wouldn't be any happier if I owned nicer things.
Misresponse to Reversed and Negated Items
Empirical data
 we analyzed items from volumes 1 through 36 of
JCR (1974 till the end of 2009) and volumes 1
through 46 of JMR (1964 to 2009);
 we included all Likert-type scales for which the items
making up the scale were reproduced in the article
and factor loadings or item-total correlations were
reported;
 total of 66 articles in which information about 1330
items measuring 314 factors was provided;
 of the 1330 distinct items in the data set, 608 came
from JCR and 722 from JMR;
Misresponse to Reversed and Negated Items
Item negation
 Items can be stated either as an assertion (affirmation) or
as a denial (disaffirmation) of something (Horn 1989);
 Negation is a grammatical issue;
 Classification of negations in terms of two dimensions:
□
what part of speech is negated (how a word is used in a
sentence: as a verb, noun/pronoun, adjective, adverb or
preposition/conjunction);
□
how the negation is achieved (by means of particle negation,
the addition of no, the use of negative affixes, negative adjectives
and adverbs, negative pronouns, or negative prepositions);
Negated by means of
Not, n’t
No
negative affixes
negative
adjectives
and adverbs
negative
pronouns
negative
prepositions
[This salesperson
does not make
false claims.]
n.a.
dislike, disagree,
etc.
[I dislike food
shopping very
much.]
reluctant,
hesitant, never,
rarely, seldom,
hardly (ever),
less, little, etc.
[I seldom
daydream.]
n.a.
without, instead
of, rather than,
etc.
[This supplier
sometimes
promises to do
things without
actually doing
them later.]
5 (1.7%)
26 (8.6%)
discomfort,
disagreement,
etc.
[There is
considerable
disagreement as
to the future
directions that
this hospital
should take.]
little, few, a lack
of, none of the,
not much, neither
of, etc.
[Many times I
feel that I have
little influence
over things that
happen to me.]
Part of speech
Verb
135 (44.7%)
Noun/
pronoun
Adjective
not everyone, not
no object, no
(only), etc.
reason, no
[I and my family
purpose, etc.
will consume
[Clipping,
only certain
organizing, and
brands, not
using coupons is
others.]
no fun.]
5 (1.7%)
Total
171 (56.6%)
no-one, nobody,
except for,
none, nothing, without, with the
etc.
exception of,
[Energy is really instead of, rather
not my problem
than, etc.
because there is [American people
simply nothing I
should always
can do about it.]
buy Americanmade products
instead of
imports.]
4 (1.3%)
17 (5.6%)
5 (1.7%)
10 (3.3%)
4 (1.3%)
14 (4.6%)
n.a.
n.a.
uninterested,
dishonest, etc.
[Most charitable
organizations are
dishonest.]
rarely, less, etc.
[I would be less
loyal to this rep
firm, if my
salesperson
moved to a new
firm.]
n.a.
n.a.
55 (18.2%)
4 (1.3%)
54 (17.9%)
59 (19.5%)
Negated by means of
Not, n’t
No
negative affixes
Part of speech
Adverb
Preposition/
Conjunction
negative
adjectives
and adverbs
not much, etc.
no longer, etc.
[I often dress
rarely, less, etc.
[After I meet
[Hard work is no unconventionally [I feel I have to
someone for the longer essential
even when it's
do things hastily
first time, I can
for the welllikely to offend
and maybe less
usually
being of society.]
others.]
carefully in order
remember what
to get everything
they look like,
done.]
but not much
about them.
2 (.7%)
1 (.3%)
3 (1.0%)
1 (.3%)
not for, not (just)
in, not (only) as,
not until, not if
(incl. unless), not
because, etc.
[I enjoyed this
shopping trip for
its own sake, not
just for the items
I may have
purchased.]
n.a.
n.a.
n.a.
10 (3.4%)
Total
151 (50.0%)
negative
pronouns
negative
prepositions
n.a.
n.a.
7 (2.3%)
for nothing, etc.
[I don't believe
in giving
anything away
for nothing.]
n.a.
11 (3.7%)
1 (.3%)
18 (6.0%)
68 (22.5%)
41 (13.5%)
Total
5 (1.7%)
19 (6.3%)
302
Misresponse to Reversed and Negated Items
Item reversal
 an item is reversed if its meaning is opposite to a
relevant standard of comparison (semantic issue);
 three senses of reversal:
□
□
reversal relative to the polarity of the construct being
measured;
reversed relative to other items measuring the same
construct:


□
reversal relative to the first item
reversal relative to the majority of the items
reversal relative to a respondent’s true position on the
issue under consideration (Swain et al. 2008);
Misresponse to Reversed and Negated Items
Item reversal (cont’d)
 in our data set of 1330 items, between 83 and 86
percent of items were nonreversed (depending on
the definition of reversal);
 the proportion of factors (or subfactors in the case of
multi-factor constructs) that do not contain reversed
items was 70 percent;
 only 8 percent of factors (out of 314 factors) were
composed of an equal number of reversed and
nonreversed items (i.e., the scale was balanced);
Misresponse to Reversed and Negated Items
Integrating item negation and item reversal
Item reversal
Non-reversed
Reversed
Regular (RG)
items
Polar opposite
(PO) items
Non-negated
Talkative, enjoying
talking to people
Quiet, preferring to do
things by oneself
Item negation
Negated polar
opposite (nPO) items
Negated
Not quiet, preferring
not to be by oneself
Negated regular
(nRG) items
Not talkative, not
getting much pleasure
chatting with people
Misresponse to Reversed and Negated Items
Cross-classification of negation and reversal
Reversal relative to
Polarity of construct
Polarity of first item
Dominant keying direction
Total
Nonreversed
Reversed
Nonreversed
Reversed
Nonreversed
Balanced
Reversed
No
negation(s)
71.1%
8.7%
71.0%
8.7%
70.3%
5.2%
4.2%
79.7%
Negation(s)
11.7%
8.7%
15.4%
4.9%
13.7%
2.9%
3.7%
20.3%
Total
82.8%
17.4%
86.4%
13.6%
84.0%
8.1%
7.9%
100.0%
Misresponse to Reversed and Negated Items
Misresponse to negated and reversed items
MR → within-participant inconsistency in response to multiple
items intended to measure the same construct;
Consistent
responding
Misresponse to
negated items
(NMR)
Misresponse to
reversed items
(RMR)
Misresponse to
polar opposites
(POMR)
Talkative (RG)
A
A
A
A
Not talkative (nRG)
D
A
A
D
Quiet (PO)
D
D
A
A
Not quiet (nPO)
A
D
A
D
Item
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Comprehension
 Careless responding (Schmitt and Stults 1985):
□
□
□
respondents fail to pay careful attention to individual
items and respond based on their overall position on
an issue [RMR];
more likely when a reversed item is preceded by a
block of nonreversed items;
Remedies:



alert Rs to the presence of reversed items, avoid
tedious surveys, and be careful with involuntary Rs;
increase Rs’ ability and avoid distractions;
use balanced scales, alternate the keying direction, and
disperse the items;
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Comprehension (cont’d)
 Reversal ambiguity:
 Rs
may not view antonyms as polar opposites [POMR];
 contradictories vs. contraries:


Antonyms can be contradictories or contraries, depending on
whether they are bounded or unbounded (Paradis and Willners
2006);
Negations of the same core concept are clearly contradictories,
but things are more complicated when the core concept differs;
 simultaneous
disagreement is more likely when items are
worded extremely (McPherson and Mohr 2005);
 “Buddhism’s ontology and epistemology appear to make
East Asians relatively comfortable with apparent
contradictions” (Wong et al. 2003, p. 86) [RMR];
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Comprehension (cont’d)
□
Remedies:




use more sophisticated procedures to identify
appropriate antonyms (formulate linguistic contrasts in
two stages; see Dickson and Albaum 1977);
may be particularly useful in cross-cultural research;
bounded antonyms have to be pretested and
unbounded antonyms have to be used with care;
extreme statements should be avoided;
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Retrieval
 Item-wording effects:
□
□
□
□
Confirmation bias (Davies 2003; Kunda et al. 1993);
Directly applicable to antonymic reversals;
For negation reversals, confirmation bias can lead to
MR if a non-negated polar opposite schema is readily
available (Mayo et al. 2004);
Remedies:


Use polar opposite reversals to get richer belief samples,
even though they may increase apparent MR;
Negation reversals have few retrieval benefits;
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Retrieval
 Positioning effects:
□
□
□
Dispersed PO items reduce carryover effects and can
increase coverage, but the task is more taxing for Rs
and internal consistency may suffer;
Item similarity may determine whether Rs engage in
additional retrieval when items are grouped together;
Remedies:


The use of dispersed antonyms should encourage the
generation of distinct belief samples;
Avoid very similar (negated) statements when items are
grouped;
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Judgment
 Item verification difficulty (Carpenter and Just 1975;
Swain et al. 2008):
□
□
MR is a function of the complexity of verifying the
truth or falsity of an item relative to one’s true beliefs,
which depends on whether the item is stated as an
affirmation or negation [NMR];
Remedies:



Negations are problematic because they increase the
likelihood of making mistakes (remember there are
many types of negations);
Negated polar opposites are most error-prone;
Mix of regular and PO reversals should be best;
Misresponse to Reversed and Negated Items
Item verification difficulty
MR
Negation
Affirmation
Truth value
True
False
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Response
 Acquiescence: Rs initially accept a statement and
subsequently re-consider it based on extant evidence; the
first stage is automatic, the second requires effort
(Knowles and Condon 1999) [RMR];
 Remedies:
□
□
Although response styles are largely individual
difference variables, situational factors may be under
the control of the researcher (e.g., reduce the
cognitive load for Rs);
Problems with online surveys;
Misresponse to Reversed and Negated Items
Theoretical explanations of MR:
Response (cont’d)
 Asymmetric scale interpretation: the midpoint of the
rating scale may not be the boundary between agreement
and disagreement for Rs (esp. if the response categories
are not labeled; cf. Gannon and Ostrom 1996) [RMR];
 Remedies:
□
Use fully labeled 5- or 7-point response scales;
Misresponse to Reversed and Negated Items
Consequences of MR
 Effect on scale reliability: [based on 4 studies]
RG .
nRG
PO
nPO
.68
.60
.57
.49
 Effect on correlations between items measuring the same
construct:
□
when content saturation is high, social desirability is low, and
endorsement frequencies are moderate, the correlation between the
four types of items can be quite high, although the findings showed
evidence of POMR (Jackson and Lay 1968);
□ Ahlawat (1985) showed that the correlations between nPO and other
items were particularly low;
Misresponse to Reversed and Negated Items
Consequences of MR (cont’d)
 Effect on factor structure: [based on three studies]
□
□
The factor structure can emerge clearly when the
items are high in content saturation and free of social
desirability, but RMR and POMR are apparent;
In Schriesheim and Eisenbach (1995), RG items
performed best and nPO items did very poorly;
 Effect on correlations with other scales:
□
Holden et al. (9185) showed that item negation was
more detrimental to criterion-related validity than item
reversal;
Misresponse to Reversed and Negated Items
Example item with 4 negations
Top management in my company has let it be known in
no uncertain terms that unethical behaviors will not be
tolerated.