Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
373
British Journal of Social Psychology (2006), 45, 373–396
q 2006 The British Psychological Society
The
British
Psychological
Society
www.bpsjournals.co.uk
Assessment of implicit personality self-concept
using the implicit association test (IAT):
Concurrent assessment of anxiousness
and angriness
Konrad Schnabel1*, Rainer Banse2 and Jens B. Asendorpf1
1
2
Humboldt-Universität zu Berlin, Germany
University of York, UK
This study explored the psychometric properties of the Implicit Association Test (IAT)
when it is employed for the assessment of two personality traits within one sample. The
sequence of an anxiousness and an angriness IAT was counterbalanced across 100
participants and the IATs’ predictive validity for anxious versus angry behaviour after
emotion inductions was examined and compared to direct self-report measures. The
anxiousness IAT added incremental validity over direct measures for the prediction of
anxious behaviour. The angriness IAT was affected by an order effect. When the
angriness IAT was completed after the anxiousness IAT both tests correlated with
r ¼ .46 whereas they were not significantly correlated when the angriness IAT was
completed first. Direct anxiousness and angriness measures were uncorrelated.
Implications for the assessment of multiple implicit personality self-concept dimensions
are discussed.
Probably the easiest way to find out how anxious individuals are is to ask them to report
their anxiousness. Consequently, direct questionnaire measures are the most commonly
used method for the assessment of personality. However, there are two reasons why
direct asking does not always provide valid information (e.g. Greenwald & Banaji, 1995).
First, individuals may try to present themselves in a favourable light by faking their
answers. For instance, they may describe themselves as self-confident although they
know that they are rather anxious. Second, individuals may not fully realize how anxious
they are due to introspective limits. They may estimate themselves as self-confident
although they show anxious reactions in many situations. In the last decades, social
cognition research developed indirect measurement procedures that are less affected by
self-presentational strategies and introspective limits, and provide complementary valid
information that can help to have a more comprehensive assessment of constructs such
* Correspondence should be addressed to Konrad Schnabel, Institut für Psychologie, Humboldt-Universität zu Berlin, Unter den
Linden 6, 10099 Berlin, Germany (e-mail: konrad.schnabel@psychologie.hu-berlin.de).
DOI:10.1348/014466605X49159
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
374
Konrad Schnabel et al.
as anxiousness. The most prominent of these procedures is the Implicit Association Test
(IAT; Greenwald, McGhee, & Schwartz, 1998).
In the IAT, the automatic association between a bipolar target concept (such as self
vs. others) and a bipolar attribute concept (such as anxious vs. self-confident) is assessed
through a series of discrimination tasks that require fast responding. Participants are
instructed to categorize exemplars of these concepts and to use two response keys for
their categorization (e.g. left response key for self and anxious words vs. right response
key for others and self-confident words). Faster responses are expected when two
highly associated categories (e.g. self and anxious for an anxious individual) share the
same response key than when they are assigned to different response keys.
The IAT proved to be a promising candidate to enrich the method spectrum of
personality assessment because of two reasons. First, several studies showed that the
IAT assesses individual differences with internal consistencies that are satisfactory
and considerably higher than these of alternative indirect procedures (see Teige,
Schnabel, Banse, & Asendorpf, 2004, for a more detailed discussion). Second, the IAT
was shown to add incremental validity over questionnaire measures for the prediction of
behaviour. Asendorpf, Banse, and Mücke (2002) showed that a shyness IAT increased
significantly the prediction of spontaneous shy behaviour in a realistic social situation.
Egloff and Schmukle (2002) showed that an anxiousness IAT but not direct self-report
measures predicted anxious behaviour during a videotaped speech. Finally, the IAT was
also shown to be useful for clinically oriented research (see De Houwer, 2002, for an
overview)
Explicit versus implicit personality self-concept
The differences between questionnaire and IAT measures or between direct and
indirect procedures at the empirical level are related to the distinction between the
explicit and the implicit personality self-concept at the construct level. In line with a
general definition of the self-concept by Greenwald et al. (2002), Asendorpf et al. (2002)
defined the self-concept of personality as an associative network containing all
associations of the concept of self with attribute concepts describing one’s personality,
thus attributes that describe individual, relatively stable, non-pathological characteristics of the person. Referring to current two system models of information processing
(e.g. Strack & Deutsch, 2004), we assume the self-concept of personality to be
represented in both explicit and implicit representations. Explicit representations of
the personality self-concept are based on propositional structures that result from
processing information in a controlled and reflective way. The proposition, ‘I am a very
anxious person’. is an example of an explicit representation of the personality selfconcept. Direct measures (e.g. questionnaires) are an attempt to assess such explicit
representations. Direct measures contain valid information as far as they refer to parts of
the self-concept of personality that are introspectively accessible. They may contain
invalid information due to self-presentational concerns or measurement error.
In contrast, implicit representations are based on associative structures that result
from processing information in an automatic and impulsive way. An example for an
implicit representation is the spontaneous tendency to associate the concept of the self
with anxiousness. Indirect measures (e.g. IAT procedures) are an attempt to assess such
implicit representations. Indirect measures are based on information that is not
intentionally given to inform about the self. They contain valid information as far as they
refer to parts of the self-concept of personality that are accessible through the particular
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
375
assessment methodology. Indirect measures may contain invalid information due to
systematic biases of the assessment methodology or measurement error.
Although we conceptualized explicit and implicit representations as propositional
and association representations, respectively, we do not assume that they are
necessarily different. Explicit and implicit representations are related to each other
because all propositional representations comprise the concepts of two or more
association representations (e.g. ‘me’ and ‘anxious’). Explicit and implicit representations are different from each other because many association representations may not
become part of propositional representations due to limited introspective accessibility.
This view stresses the interactions between explicit and implicit representations
as components of a reflective and an impulsive system of information processing
(cf. Strack & Deutsch, 2004). Consequently, there is no pure measure of the explicit or
the implicit personality self-concept. However, we assume that indirect procedures like
the IAT primarily involve the impulsive system (Asendorpf et al., 2002) and refer only
secondarily to controlled processes within the reflective system (cf. Fazio & Olson,
2003). Therefore, indirect procedures aim at measuring implicit representations and
should show incremental validity over direct measures that aim at assessing explicit
representations.
Goals of the present research
The main goal of the present study was to test whether the IAT allows for the assessment
of more than one personality attribute within one testing session. Using the traits of
anxiousness and angriness as examples, we explored whether the IAT shows potential
for the assessment of multiple personality dimensions. The traits were labelled
‘anxiousness’ and ‘angriness’ rather than ‘anxiety’ and ‘anger’ to make clear that they
refer to personality traits and not to emotional states. Throughout this article, we use the
terms ‘anxiousness’ and ‘angriness’ when we refer to trait measures (i.e. measures that
assess how people usually feel) and the terms ‘anxiety’ and ‘anger’ when we refer to
state measures (i.e. measures that assess how people actually feel). Direct measures
offer broad possibilities for the assessment of multiple personality traits. For instance,
the Big Five measures (e.g. Costa & McCrae, 1992) allow for the assessment of five
relatively independent personality dimensions. Anxiousness and angriness are
differently related to the three Big Five dimensions neuroticism, extraversion, and
agreeableness. Anxiousness is highly correlated with neuroticism, moderately
correlated with introversion, and uncorrelated with agreeableness. In contrast,
angriness is highly negatively correlated with agreeableness and is weakly positively
correlated with both neuroticism and extraversion (Ostendorf, 1990). In accordance
with their opposite correlation patterns, anxiousness and angriness are conceptualized
as orthogonal dimensions and are uncorrelated. This pattern of relationships facilitates
the study of convergent and discriminant validity between direct and indirect measures
because zero correlations are expected for all correlations between anxiousness and
angriness. Furthermore, a correlation between the anxiousness and the angriness IAT
can be interpreted as shared method variance.
The validity of the anxiousness and the angriness IAT should not only be explored as
convergent and discriminant validity with direct measures, but also as predictive validity
for the prediction of anxious and angry behaviour after emotion inductions. This is
especially interesting because previous research demonstrated that the anxiousness IAT
shows incremental validity over direct measures for the prediction of anxious behaviour
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
376
Konrad Schnabel et al.
(Egloff & Schmukle, 2002). Thus, the present study aimed to test the incremental
validity of the IAT when it is used for the concurrent assessment of anxiousness and
angriness. The anxiousness and the angriness IAT were applied as two consecutive tests
and their sequence was counterbalanced across participants to control for order effects.
Methods
Participants
A total of 103 university students were recruited as participants on the campus of
Humboldt University, Berlin, none of whom were psychology students or had
participated in the laboratory’s previous studies. Most participants were directly
approached by an experimenter (not identical with the experimenter at the laboratory).
Some participants were recruited using postings at the university buildings. Participants
were asked to take part in a study on concentration and personality. As a compensation,
participants were offered e10 (approximately £7 at the time) for completing a
questionnaire of about 15 minutes duration at home and for participating in a laboratory
experiment of about one hour duration. In addition, they could receive individual
feedback on their results after the study was complete. All participants claimed to be
native German speakers. Three female participants refused to complete the speaking
task during the laboratory session, and were therefore excluded from analysis. This led
to a final sample of 100 participants (50 male, 50 female; age M ¼ 24.0 years, range
19–32 years).
Assessments and measures
Overall procedure and design
All participants (a) judged themselves on several personality measures at home one
week before the laboratory session. After arrival at the laboratory, they (b) completed a
short form of the d2 Attention-Stress Test, (c) completed the anxiousness IAT and the
angriness IAT, (d ) indicated their state anxiety and state anger on bipolar items,
(e) received instructions for an anxiety-inducing speech, (f ) completed a retest of (d ),
(g) prepared their speech, (h) were videotaped during their speech, (i) were videotaped
during an anger-inducing computer crash, ( j ) completed a retest of (d), ( k) were
interviewed about the experiment, and ( l) were completely debriefed.
The anxiousness and angriness items of the two IATs were included as direct selfratings in step (a), (d ), (f ) and ( j ). The order of the anxiousness IAT and the angriness
IAT in step (c) was varied between participants such that half of the participants
completed the anxiousness IAT first and the other half completed the angriness IAT first.
The assignment to the two orders was balanced for gender and alternated between
successive participants. In contrast, the order of the anxiety and the anger induction
was fixed, such that the anxiety induction always came first, because it seemed difficult
to successfully induce anxiety after the faked computer crash.
Finally, the participants were thanked and asked to give their consent for the
evaluation of the videotapes (all agreed). They were also paid and promised individual
feedback about their results. Four months after finishing data collection, participants
received a letter containing the principal findings of the study along with an invitation
for an individual feedback session, in which interested participants were informed about
their personal results.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
377
Trait measures
In order to minimize transfer effects between direct and indirect measures, direct trait
measures were mailed to the participants at least 1 week before the laboratory session.
The instructions explained to participants that the study was about concentration and
personality and consisted of two parts: a set of questionnaires concerning several
personality attributes, that was attached and had to be completed at home, and a
subsequent laboratory session assessing attention and concentration. We avoided telling
participants that the study was about anxiousness and angriness because we (a) did not
want anxious persons to avoid participation in the study, and (b) wished keep
participants naive about the anger induction, as most people would not get angry
knowing that it was intended to provoke their anger (Stemmler, Heldmann, Pauls, &
Scherer, 2001).
The mailed questionnaire contained the following measures (test references list the
used German version first, and the English equivalent second, if such equivalent
existed). The questionnaire started with the trait forms of the State Trait Anxiety
Inventory (STAI; Laux, Glanzmann, Schaffner, & Spielberger, 1981; Spielberger,
Grousch, & Lushene, 1970) and the State Trait Anger Expression Inventory (STAXI;
Schwenkmezger, Hodapp, & Spielberger, 1992; Spielberger, 1988) together with the
subscales ‘interference’ and ‘lack of confidence’ (without the item ‘Ich bin überzeugt,
dass ich gut abschneiden werde’ [‘I am sure, that I will receive good marks’]) of the Test
Anxiety Inventory (TAI-G; Hodapp, 1991; expanded German version of the TAI,
Spielberger, 1980). These questionnaires assess enduring symptoms of anxiousness,
angriness, and test anxiousness on a 4-point scale (1 ¼ almost never, 4 ¼ almost
always) with 20, 10, and 11 items, respectively. The TAI-G subscales were added, and all
scales were mixed in a fixed random order, because some participants in a pilot study
became suspicious about the cover story when the STAI and the STAXI were presented
in separate blocks. When both scales were mixed with the TAI-G, the STAI, and the
STAXI were less salient, and the true content of the experiment was much less apparent.
The trait measures proceeded with the second series of the Speaking anxiety scale
(Spitznagel, Schlutt, & Schmidt-Atzert, 2000). This questionnaire assesses habitual
emotionality (e.g. ‘I am quite nervous’) and worries (e.g. ‘I fear negative consequences’)
immediately before giving a speech with 8 items each. Items were presented on a
4-point scale (1 ¼ I do not agree at all, 4 ¼ I agree completely).
Subsequently, participants had to rate their conscientiousness, intellect,
attentiveness, anxiousness and angriness on 33 bipolar adjective pairs (e.g. ‘selfconfident 1-2-3-4-5-6-7 anxious’). The pairs were mixed in a fixed random order and
presented with a trait instruction. The 10 intellect and 10 conscientiousness pairs
were the same as in Asendorpf et al.’s Study 1 (2002). We further added 3 attentiveness
pairs to make the cover story more plausible. The first pair was ‘aufmerksam’
[‘attentive’] versus ‘durcheinander’ [‘jittery’] that was adapted from the positive and
negative affect schedule (PANAS; Krohne, Egloff, Kohlmann, & Tausch, 1996; Watson,
Clark, & Tellegen, 1988). Two additional pairs were synonymous.
The five anxiousness pairs (anxious vs. self-confident) and the five angriness pairs
(angry vs. self-controlled) were constructed on the basis of 430 unipolar and 179 bipolar
adjective items provided by Ostendorf (1990). He had factor analysed these adjective
items and reported their loadings on the first five factors that could be interpreted as the
factors of the five-factor model of personality. For the anxiousness pairs, we selected
adjective items that were strongly correlated with neuroticism, moderately correlated
with introversion, and uncorrelated with agreeableness. For the angriness pairs, we
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
378
Konrad Schnabel et al.
selected items that were weakly correlated with neuroticism and extraversion, and
strongly negatively correlated with agreeableness. These items were pre-tested in a
student sample (N ¼ 42; age M ¼ 22.6 years, range 19–39 years). The resulting 5-item
bipolar anxiousness scale showed acceptable internal consistency (a ¼ .84), correlated
strongly with neuroticism (r ¼ .82; p , .001), intermediately with extraversion
(r ¼ 2.45; p ¼ .003), and non-significantly with agreeableness (r ¼ 2.19).
The resulting 5-item bipolar angriness scale showed acceptable internal consistency,
a ¼ .77 and correlated marginally with neuroticism (r ¼ .21; p ¼ .18) and extraversion
(r ¼ .22; p ¼ .17), highly with agreeableness (r ¼ 2.78; p , .001), and was not
correlated with the 5-item anxiousness scale (r ¼ .01). All items of the bipolar
anxiousness and angriness scale were used as stimulus words for the IATs and are listed
in Table 1.
Finally, the questionnaire concluded with the social desirability scales by Lück and
Timaeus (1969; Crowne & Marlowe, 1960) and Stöber (1999; without the item, ‘Have
you ever consumed drugs’). These scales contain 16 and 23 items, respectively, and
measure socially desirable responding by asking for socially desirable but infrequent or
socially undesirable but frequent behaviours on a true-false format. Items of both scales
were presented in a fixed random order together with the Manifest anxiety scale
(MAS; Lück & Timaeus, 1969; Taylor, 1953). The 23 items of this scale assess various
symptoms of anxiousness (e.g. ‘I work under a great deal of tension’). The reliability of
all trait measures was satisfactory and is reported in Table 3 of the Results section.
Laboratory session
Upon arrival at the laboratory participants were reminded that the experiment was
about attention and concentration. The experimenter briefly explained that the
laboratory session contained different concentration tests, two of which were on the
computer, and one being a paper-and-pencil test, as well as a situation demanding
attention and concentration that would be videotaped. Subsequently, participants
received instructions for the first concentration test. Because men might repress their
anger facing a woman, and women might avoid getting angry with a physically superior
man, the experimenter was always of the same gender as the participant.
d2 test
As first test, participants completed a shortened two-minute version of the d2 Test of
Attention (d2 Test; Brickenkamp, 1994). The d2 Test is a paper-and-pencil test that asks
participants to perform a simple discrimination task by crossing out as fast as possible
relevant stimuli (the letter d with two lines above or below it) while ignoring irrelevant
stimuli (the letter d with more or fewer than two lines as well as the letter p). We used
the d2 Test to give a better justification for the cover story; therefore, results will not be
reported here.
Anxiousness and angriness IAT
Task sequence and stimuli of the anxiousness and the angriness Implicit Association Test
(IAT) are depicted in Table 1. Since this study focused on inter-individual differences,
and because we did not want to confound person variance with method variance,
the task sequence and the stimulus order was fixed within both IATs. Both IATs were the
same, except for the attribute dimension, being anxious versus self-confident within
40
40
80
40
80
Others
They
Them
Your
You
Other
1
2
3
4
5
Me
I
Self
My
Me
Own
anxious
timid
insecure
worried
overly cautious
Anxious
self-confident
daring
secure
unconcerned
carefree
Self-confident
Stimuli
Me
Anxious (angry)
Me, anxious (angry)
Others
Others, anxious (angry)
Left key
Anxiousness IAT
Target discrimination
Attribute discrimination
Initial combined task
Reversed target discrimination
Reversed combined task
Task
angry
hot-tempered
undercontrolled
hot-headed
Irritable
Angry
Angriness IAT
Others
Self-confident (self-controlled)
Others, self-confident (self-controlled)
Me
Me, self-confident (self-controlled)
Right key
self-controlled
thoughtful
self-disciplined
adaptable
calm
Self-controlled
Note. IAT ¼ Implicit Association Test. The procedures of the anxiousness and the angriness IAT were identical. Words in parentheses refer to the task sequence
within the angriness IAT. The original German stimuli are depicted in Table 4.
N of trials
Sequence
Response key assignment
Table 1. Implicit Association Tests for anxiousness and angriness: Task sequence and stimuli
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
379
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
380
Konrad Schnabel et al.
the anxiousness IAT, and angry versus self-controlled within the angriness IAT. Each IAT
consisted of five different blocks of discrimination tasks. On each trial, a stimulus word
was displayed in the centre of the screen. Participants were instructed to categorize the
stimulus as quickly and accurately as possible according to the category labels that were
displayed in the right or left upper screen corner. The category labels were assigned to
the right (the number ‘5’ of the numeric keypad) or left (the letter ‘a’) response key,
respectively. Responses were recorded using ERTS software (Behringer, 1994). After
correct responses, the inter-stimulus interval was 300 ms. After incorrect responses, the
stimulus was immediately replaced by the word fehler (German for error) for 1,000 ms,
resulting in a 1,300 inter-stimulus interval. In the combined tasks, the stimuli alternated
between target (me, others) and attribute (anxious, self-secure or angry, self-controlled)
discrimination.
Data reduction procedure followed Greenwald, Nosek, and Banaji (2003) and IAT
scores were computed as D measures with an error penalty of 600 ms, and without the
exclusion of trials below 400 ms (for details on the complete algorithm, see Greenwald
et al., 2003). As with conventional scores, D measures were based on the difference
between mean response latencies in the combined blocks (Sequence 5 and Sequence 3;
see Table 1), but were scaled in units of the individuals’ standard deviations and included
an error penalty for incorrect responses. High scores represented quicker associations
of me-anxious and others-self-confident relatively to me-self-confident and othersanxious, or of me-angry and others-self-controlled relatively to me-self-controlled and
others-angry, respectively. In contrast to Greenwald et al., all trials were considered
equally and the first 20 trials were not weighed as more important as the succeeding
trails, because we (a) did not declare the first 20 trials as training trials and (b) had 60
instead of 40 succeeding trials. (Different procedures of weighing the first 20 trials
more than the succeeding trials did only minimally change the results). Internal
consistencies were evaluated across test halves and are reported in the Results section.
State measures
As a manipulation check for the emotion inductions, we used bipolar items for
anxiousness and angriness together with a state instruction. These items were mixed in
a fixed random order with 3 out of 10 attentiveness and 7 out of 10 conscientiousness
items. The items were presented in a paper-pencil version, and were identical to those
completed as a trait measure at home. Three conscientiousness items were dropped,
because we expected them not to match the state instruction (e.g. ‘fleibig’
[‘industrious’] vs. ‘faul’ [‘lazy’]). State measures were presented after the IATs
(baseline), the instructions for the speech (anxiety induction), and after the computer
crash (anger induction). Reliabilities for the state measures were satisfactory, internal
consistencies were for the anxiety scale a ¼ .89, for the change in anxiety (speech
minus baseline) a ¼ .78, for the anger scale a ¼ .80, and for the change in anger
(computer crash minus baseline) a ¼ .74.
Anxiety induction
Participants received instructions for the speech on a piece of paper. The paper informed
participants that they should give a speech that would be videotaped and later on
analysed by experts. The requested duration of the speech was 5 minutes. Directly after
this announcement, participants completed the state measures. Subsequently, they were
told about the subject of the speech (terminal illness and euthanasia: immoral or humane;
adapted from Egloff, Schmukle, Burns, & Schwerdtfeger, in press, Study 3) and were given
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
381
3 minutes for preparation. Participants were allowed to make notes during preparation,
but the speech was supposed to be given without notes. Then, participants gave their
speech directly in front of the video-camera that was operated by the experimenter from a
nearby room. After exactly 5 minutes, the experimenter thanked the participants and
informed them that this was enough. When participants stopped talking before the 5
minutes were over, the experimenter prompted them to continue talking until full 5
minutes were up. The time period before participants continued their speech was
defined as missing. For the judgments and codings of anxious behaviour, secondary tapes
were prepared that contained the first 3 minutes of noninterrupted speech. The speech
task was followed by the anger induction.
Anger induction
The general procedure was adapted from Wiedig (2004) and is similar to a procedure
used by Bargh, Chen, and Burrows (1996, Experiment 3). Participants completed a
STROOP-Test on the computer. Again, participants were videotaped and were told that
this was to evaluate their eye-blink-rate as an indicator of concentration. In fact, this was
to give good reason for videotaping the interaction with the experimenter. Three
minutes after starting the STROOP, the screen froze and the words ‘FATAL ERROR’
appeared in the centre of the screen. In addition, a short but intensive error sound was
given, whenever a key was pressed. The experimenter then approached the participant
and pretended to be astonished by the accident. The subsequent interaction between
experimenter and participant comprised three different provocations. First, the
experimenter accused the participant of causing the crash by incorrectly using the enter
key. Second, she or he said that all computer-based data of the participant were now
destroyed. Third, due to loss of data, participants could not receive any money for the
experiment. After this, participants were asked to complete the state measures, waiting
for a computer expert who may help to save the data. For the judgments and codings of
angry behaviour, secondary tapes were prepared. The recording started when the
computer crashed and ended when participants began completing the state measures.
For the anger judgments, a 3-second blue screen interval was inserted after the end of
each of the three provocations to enable separate ratings for each provocation.
Interview
The aim of the interview was to identify participants who doubted the alleged computer
crash. Participants were asked the following questions: (1) ‘Did you have difficulties
with any part of the experiment?’, (2) ‘Did you notice anything remarkable during the
experiment?’, (3) ‘Did anything in the experiment seemed strange to you?’ (4) ‘What did
others tell you about the experiment?’ and (5) ‘What do you think the experiment was
about?’ All participants (11 female and 12 male) who mentioned in response to any of
these questions that the computer crash was part of the experiment were excluded
from the analysis of the anger induction. These participants did not differ significantly
from the remaining participants on any of the anxiousness and angriness measures.
Debriefing
Finally, participants were completely and thoroughly debriefed about the true purpose
of the study. It was ensured that participants had an opportunity to relax after the
disturbing computer crash, and would not leave the laboratory angry or upset. In the
beginning of debriefing, the participants were offered some sweets by the experimenter
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
382
Konrad Schnabel et al.
as a compensation for a rather harsh preceding interaction. Then, participants were
informed that the study was not on concentration and attention but on anxiousness and
angriness, and aimed to validate new computer-based measures for these traits. Thereby,
the experimenter went through the crucial parts of the study (direct and indirect
measures, emotion inductions) and explained why these procedures were designed to
assess anxiousness and angriness. In order to keep the true purpose of the study
undisclosed for the subsequent participants, the experimenter asked the participants to
keep the information about the study confidential until they received a letter from the
experimenter. This letter was sent out 4 months after finishing data collection and
comprised the main findings of the study together with an invitation for an individual
feedback session.
Judgments of anxious and angry behaviour
Four student judges who were unfamiliar with the participants and blind to their data
independently rated their overall impression of the participants’ anxiety (1 ¼ ‘not at
all anxious’, 7 ¼ ‘very anxious’) and anger (1 ¼ ‘very angry’, 7 ¼ ‘not at all angry’) on
7-point scales. For the anxiety judgment, three consecutive 1-minute intervals of the
3-minute speech were rated. For the anger judgment, each of the three provocations
after the computer crash (alleged misuse of the enter key, loss of data, no money)
was judged separately. The resulting 12 anxiety and 12 anger judgments were
averaged for each participant. The anxiety ratings were anchored by a female and a
male example of extremely anxious and extremely nonanxious participants from the
study by Egloff and Schmukle (2002). In the same way, the anger judgments were
anchored by extremely angry and non-angry examples from the study by Wiedig
(2004). Inter-rater reliability was satisfactory for all judgments (see Results section).
Results
Efficacy of emotion inductions
To investigate whether the speech and the computer crash were apt to observe
anxious and angry behaviour, we first examined the efficacy of these emotion
inductions. One-way ANOVAs for repeated measures were conducted to estimate
differences of self-reported state anxiety and state anger across the three measurement
points (baseline, announcement of the speech, computer crash). Results showed that
state anxiety differed significantly, F(2, 198) ¼ 14.12, p , .001) and state anger differed
marginally, F(2, 152) ¼ 2.77, p , .10) across the three situations. (The degrees of
freedom were smaller for anger because we had to exclude participants who were
suspicious about the computer crash). Single comparisons (one-sided tests) showed
that participants reported more state anxiety after the announcement of the speech
(M ¼ 3.39, SD ¼ 1.04) than at the beginning of the experiment (M ¼ 3.02, SD ¼ 0.81),
t(99) ¼ 4.11, p , .001, d ¼ .58, and more state anxiety after the announcement of the
speech than after the computer crash (M ¼ 2.92, SD ¼ 0.84), t(99) ¼ 4.67, p , .001,
d ¼ .66. Baseline anxiety and state anxiety after the computer crash did not differ
significantly from each other, t(99) ¼ 1.14, ns. Concerning changes in state anger,
participants reported more state anger after the computer crash (M ¼ 2.53, SD ¼ 0.75)
than at the beginning of the experiment (M ¼ 2.35, SD ¼ 0.71), t(76) ¼ 1.98, p , .05,
d ¼ .31. However, state anger was also higher after the announcement of the speech
than at baseline, t(76) ¼ 2.37, p , .05, d ¼ .38, and participants did not report more
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
383
state anger after the computer crash than after the announcement of the speech
(M ¼ 2.49, SD ¼ .71), t(76) ¼ .50, ns. Thus, only the announcement of the speech, but
not the computer crash, increased self-reported state anxiety, whereas both the
computer crash and the announcement of the speech tended to increase self-reported
state anger. Because we did not counterbalance the sequence of the emotion
inductions, and the speech situation was always before the faked computer crash (see
Methods section), we could not examine whether state anger would have been
increased by the computer crash alone. Even though the anger induction effect was
relatively small, the significant correlations between direct angriness measures and the
observer judgments of angry behaviour (see below) are an indicator for the validity of
the anger induction.
Descriptive statistics for the anxiousness IAT, the angriness IAT, and the observer
judgments
The descriptive statistics of the IATs and the observer judgments are presented in
Table 2. As can be seen in the table, the mean raw scores were negative for both IATs. In
the anxiousness IAT, only 9 (6 female, 3 male) out of 100 participants had positive IAT
scores. Thus, most of the participants were quicker to combine me þ self-confident
and others þ anxious than for the reverse mapping. In the angriness IAT, only 4
(1 female, 3 male) out of 100 participants had positive scores. Thus, most of the
participants were quicker to combine me þ self-controlled and others þ angry than
for the reverse mapping. Mean error rates were for the anxiousness IAT, M ¼ 4.2%,
SD ¼ 2.6%, and for the angriness IAT, M ¼ 3.6%, SD ¼ 2.3%. In any IAT, no participant
had error rates higher than 15% or more than 10% of the latencies faster than 300 ms.
The distributions of the improved and individually standardized D measures were not
even marginally different from a normal distribution in both IATs, Z , 1. Internal
consistency (see Table 2) for the IATs was computed across the two test halves and was
acceptable for the anxiousness IAT but somewhat unsatisfactory for the angriness IAT.
Internal consistency was satisfactory for the observer anxiety and angry judgments.
Convergent and discriminant validity of direct measures
This section inspects the convergent and discriminant validity of the bipolar
anxiousness and angriness self-ratings that were also used as word material within the
Table 2. Descriptive statistics of the IATs and the observer judgments
Variables (range of scores)
Na
M
SD
Range
Reliabilityb
Anxiousness IATc
Angriness IATc
Observer anxiety judgment (1–7)
Observer anger judgment (1–7)
100
100
100
77
2 171.1
2 186.6
3.27
3.80
156.9
133.2
1.06
.83
2 641–179
2 533–161
1.33–6.42
1.75–6.08
.72
.66
.89
.87
Note. IAT ¼ Implicit Association Test.
a
Sample size is smaller for the observer anger judgment because participants, who realized that the
anger induction was part of the experiment, had to be excluded from the analyses of the anger
induction.
b
Internal consistency alpha for IATs; agreement a of four observers for observer judgments.
c
In milliseconds except for reliability.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
384
Konrad Schnabel et al.
IATs. The reliabilities and correlations of all direct trait measures are depicted in Table 3.
Reliability (Cronbach’s a) was satisfactory for all measures, in particular, it was .84
for the bipolar anxiousness and .80 for the bipolar angriness self-rating. As can be seen in
the first two rows of Table 3, the bipolar anxiousness self-rating correlated highly with
the Manifest anxiety scale and the trait form of the STAI, and intermediately with the
subscales of the Speaking anxiety scale. These subscales assess habitual emotionality
and worries immediately before giving a speech and, in contrast to general anxiousness
questionnaires, are more situation-specific. The bipolar anxiousness self-rating also
showed a low correlation with the trait form of the STAXI.
In contrast, the bipolar angriness self-rating did not even marginally correlate
with any direct anxiousness measure and correlated intermediately with the trait
form of the STAXI. Thus, the correlation for the angriness self-rating with the
corresponding trait measure was somewhat lower than for the anxiousness selfrating. Nevertheless, a Steiger’s (1980) test of correlation differences revealed that the
bipolar angriness self-rating correlated marginally higher with the trait form of the
STAXI, r ¼ .45, than the bipolar anxiousness self-rating, r ¼ .23, t(97) ¼ 1.65, p ¼ .05
(one-tailed). Moreover, the trait form of the STAXI did not only correlate with the
bipolar anxiousness self-rating, but also with other direct anxiousness measures. This
replicated the results of previous studies (Schwenkmezger et al., 1992) showing that
the STAXI was correlated with anxiousness because individuals high in neuroticism
were more concerned with their anger expression than those individuals who were
emotionally stable. The lack of discriminant validity of the trait form of the STAXI
may further account for the intermediate correlation between this scale and the
bipolar angriness self-rating. Thus, convergent and discriminant validity with
established measures were shown for the bipolar anxiousness and angriness selfratings. This validated the word material we used as attributes within the IATs, at
least at the level of direct measures. Importantly, bipolar anxiousness and angriness
self-ratings were uncorrelated (r ¼ .08, ns) as it was expected because they were
conceptualized as orthogonal dimensions.
Convergent and discriminant validity of indirect measures
As can be seen in Table 3, the anxiousness IAT correlated significantly with the bipolar
anxiousness self-rating and the Manifest anxiety scale, and marginally with the worries
subscale of the Speaking Anxiety Questionnaire and the trait form of the STAI.
The anxiousness IAT did not correlate with the bipolar angriness self-rating or the trait
form of the STAXI. The angriness IAT, in contrast, correlated neither with direct angriness
nor with direct anxiousness measures. Thus, discriminant and convergent validity
with direct measures was shown for the anxiousness IAT but not for the angriness IAT.
Surprisingly, the correlation between the anxiousness and the angriness IAT was
significantly positive (r ¼ .32, p , .01), although direct anxiousness and angriness
measures as well as the observer anxiety and anger judgments were uncorrelated
(see Table 3). Moreover, the presentation order of the IATs moderated the
correlation between the IATs. In the group that completed the anxiousness IAT
first, both IATs were substantially correlated, r ¼ .49, p , .001, whereas they were
not even marginally correlated in the group that completed the angriness IAT first,
r ¼ .17, ns. This correlation difference was marginally significant, z ¼ 1.77, p , .10
(two-tailed) and should not be attributed to sample effects, because direct
anxiousness and angriness were uncorrelated in both groups of different IAT order.
2
3
4
.72***
.12
.36***
.44***
.82
5
.73***
.07
.28**
.40***
.78***
.90
6
.23*
.45***
.13
.23*
.39***
.37***
.78
7
9
10
11
12a
2.08
.25* 2.04
.22* 2.04
2.30** 2 .03
.11
2 .01
.38**
2.16
2 .01 2.03
.29**
.03
2.23*
.17þ
.05
.15
.02
2.30**
.21*
.00
.19þ
.04
þ
2.25*
.17
.02
.19þ
.06
2.34*** 2 .01
.03
.00
.33**
.81
.02 2.08
.06
2.05
.72
.32**
.26** 2.09
.66 2 .07
2.11
.89
.00
.87
8
Note. IAT ¼ Implicit Association Test. N ¼ 100. Internal consistencies (Cronbach’s a) are printed in italics along the diagonal.
þ
p , .05; *p , .05; **p , .01; ***p , .001.
a
N ¼ 77.
b
Trait form.
1. Bipolar anxiousness self-rating
.84 2 .08
.30**
.35***
2. Bipolar angriness self-rating
.80 2 .05
2.05
3. Speaking anxiety emotionality
.88
.72***
4. Speaking anxiety worries
.84
5. Manifest anxiety scale
6. State trait anxiety inventoryb
7. State trait anger expression inventoryb
8. Social desirability
9. Anxiousness IAT
10. Angriness IAT
11. Observer anxiety judgment
12. Observer anger judgmenta
13. State anxiety after emotion induction
14. State anger after emotion inductiona
1
Table 3. Correlations between the main variables
.56***
2.02
.52***
.40***
.47***
.45***
.09
2.18þ
.11
2.08
.38***
2.12
.89
13
.02
.18
2 .06
.03
.13
.09
.17
2 .06
.16
2 .06
2 .01
.14
2 .07
.80
14a
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
385
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
386
Konrad Schnabel et al.
Importantly, this order effect was recently replicated in an independent study (Teige
et al., 2004).
A possible explanation for this asymmetrical effect (the IATs were correlated when
the anxiousness IAT was the first test but not when the angriness IAT was the first test)
might be that a positive-negative or valence dimension was stronger in the anxiousness
IAT than in the angriness IAT. Working on the anxiousness IAT, participants could have
possibly developed a classification heuristic, discriminating anxious versus selfconfident as positive versus negative attributes. In other words, participants may have
recoded the IAT task, because a discrimination of positive versus negative is easier than a
discrimination of anxious versus self-confident (cf. De Houwer, 2001). This taskrecoding was salient during the anxiousness IAT. After completion of the anxiousness
IAT, the task recoding could have been transferred onto the angriness IAT, which would
have lead to a positive correlation between both IATs. In contrast, the angriness IAT was
less likely to elicit a positive-negative task recoding, because angry versus self-controlled
is less associated with a positive-negative dimension. Consequently, when the angriness
IAT was the first test, the participants did not spontaneously use a positive-negative
recoding of the task, and the IATs did not correlate with each other.
To examine whether the anxiousness and the angriness IAT differed with respect to a
positive-negative dimension, 41 undergraduate psychology students rated the valence of
the IAT stimuli on a 7-point scale (negative [2 2 2 ] [2 2 ] [2 ] [0] [þ ] [þ þ ] [þ þ þ ]
positive). Answers were coded such that higher values indicated more positive valence.
Means and standard deviations of the valence ratings are shown in Table 4. Results
showed that the five self-confident attributes were judged more positively than the five
self-control attributes, t(40) ¼ 6.82, p , .001, d ¼ 1.50, whereas the five anxious
attributes were not judged more negatively than the five angry attributes, t(40) ¼ 1.35,
Table 4. Valence ratings of the IAT stimuli
Attributes
M
SD
Range
Attributes
Anxious (ängstlich)
Timid (furchtsam)
Insecure (unsicher)
2.68 1.15
2.49 1.08
2.20 0.84
1–6
1–5
1–4
Angry (ärgerlich)
Hot-tempered (aufbrausend)
Undercontrolled
(unbeherrscht)
Hot-headed (hitzköpfig)
Irritable (motzig)
Worried (besorgt)
Overly cautious
(übervorsichtig)
3.78 1.44
2.17 1.00
1–6
1–5
Mean anxious attributes
2.66 0.83 1.2–5.0 Mean angry attributes
Self-controlled (kontrolliert)
Thoughtful (bedächtig)
Self-disciplined
(selbstbeherrscht)
Adaptable (fügsam)
Calm (friedlich)
M
SD
Range
3.07 1.27
2.34 1.28
1.88 0.87
1–6
1–6
1–4
2.83 1.30
1.95 1.09
1–7
1–6
2.41 0.70 1.4–4.6
Self-confident (sicher)
6.02 0.82
Daring (wagemutig)
4.85 1.20
Secure (selbstvertrauend) 6.37 0.66
4–7
3–7
5–7
4.76 1.37
4.73 0.92
4.98 1.19
2–7
3–7
3–7
Unconcerned (sorglos)
Carefree (unbeschwert)
4.24 1.56
5.44 1.23
1–7
2–7
2.46 1.16
5.54 1.05
1–6
3–7
Mean self-confident
attributes
5.39 0.69 3.6–6.6 Mean self-controlled attributes 4.49 0.61 3.4–6.0
Note. IAT ¼ Implicit Association Test. The scale format was a 7-point scale with 1 indicating negative,
4 indicating neutral, and 7 indicating positive valence.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
387
p ¼ .18, d ¼ .31. Thus, we found the expected difference between the stimuli of
the anxiousness and the angriness IAT with respect to valence, but it was true only for the
positive attributes (i.e. the self-confident and the self-control attributes) and not for the
negative attributes (i.e. the anxious and the angry attributes). Importantly, one attribute
(adaptable) within the self-control attributes was judged negatively when it was tested
against the neutral scale mid-point, t(40) ¼ 8.45, p , .001, whereas none of the anxious
attributes was judged positively, and none of the self-confident attributes was judged
negatively. In sum, an underlying positive-negative dimension was less apparent in the
angriness IAT than in the anxiousness IAT. Consequently, there might have been the
transfer of a positive-negative dimension from the anxiousness IAT on the angriness IAT,
but not vice versa.
Predictive validity of direct and indirect measures
In this section, we report the results of hierarchical regression analyses that explored
whether the observer anxiety and anger judgments were predicted by direct and
indirect measures. Because we wanted to study the incremental validity of the IATs
over and above direct measures, we entered the anxiousness or angriness IAT after the
direct measures. We also performed preliminary tests and explored whether the order
of the IATs moderated the predictive validity of the IATs, entering both the order main
term and the interaction terms into the regressions. Neither the order main term nor
the interaction terms were significant. Thus, we did not introduce the interaction terms
in the final regressions but we entered the order main term in Step 1 to covariate order
effects from the results. To examine the prediction of the observer anxiety judgment,
we entered the order effect in Step 1, all direct anxiousness and anxiety measures
(the bipolar anxiousness self-rating, the subscales emotionality and worries of the
Speaking Anxiety Questionnaire, the trait form of the STAI, the Manifest anxiety scale,
and the bipolar state anxiety self-rating) in Step 2, and the anxiousness IAT in Step 3.
The direct measures contributed significantly to the prediction of the observer anxiety
judgment when entered in Step 2, R 2 ¼ .142, p , .05, and the anxiousness IAT
showed an independent contribution when entered in Step 3, R 2change ¼ .064, p , .01.
When all variables were entered into the equation in Step 3, the bipolar state anxiety
self-rating and the anxiousness IAT were significant predictors, b ¼ 0.28, t ¼ 2.28,
p , .05, b ¼ 0.27, t ¼ 2.81, p , .01, the emotionality subscale was a marginal
predictor, b ¼ 0.29, t ¼ 1.96, p , .10, and all other predictors were not significant.
To examine the prediction of the observer anger judgment, we carried out the same
hierarchical regressions. Again, the presentation order of the IATs was entered in Step 1,
direct measures (the bipolar angriness self-rating, the trait form of the STAXI, and the
bipolar state anger self-rating) were entered in Step 2, and the angriness IAT was entered
in Step 3. The direct measures contributed significantly to the prediction of the observer
anger judgment when entered in Step 2, R 2 ¼ .180, p , .01. However, the angriness
IAT did not show an independent contribution when entered in Step 3, R 2change ¼ .017,
ns. When all variables were entered into the equation in Step 3, the bipolar angriness
self-rating was a significant predictor, b ¼ 0.30, t ¼ 2.60, p , .05, the trait form of the
STAXI was a marginally significant predictor, b ¼ 0.20, t ¼ 1.69, p , .10, and the state
anger self-rating and the angriness IAT did not significantly account for the observer
anger judgment, b ¼ 0.05, t ¼ .40, ns, b ¼ 0.13, t ¼ 2 1.24, ns.
In order to explore whether the lack of predictive validity of the angriness IAT may
be attributed to the relatively small anger induction effect (d ¼ .31) we conducted
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
388
Konrad Schnabel et al.
High
Figure 1. Graphical representation of the interaction effect between the angriness IAT and selfreported state anger in predicting angry behaviour.
a moderator analysis. We regressed the observer anger judgment on the angriness IAT,
on self-reported state anger after the computer crash (residuals of a simple regression on
baseline anger), and on the interaction term of both variables. The statistic of interest
was the regression weight of the interaction term. Surprisingly, it was negative,
b ¼ 0.28, t ¼ 2 2.49, p , .05, R 2change ¼ .075. The interaction is displayed graphically in
Figure 1. As indicated by the regression slopes, the correlation between the angriness
IAT and the observer anger judgment tended to be positive for participants reporting
low (median split) state anger and negative for participants reporting high state anger,
r ¼ .12, ns, N ¼ 38 and r ¼ 2.28, p , .10, N ¼ 39, respectively.
We have no plausible explanation why the IAT-behaviour correlation tended to be
negative for participants reporting high state anger. Additionally, this moderation effect
was not true for the direct angriness measures. Thus, we refrain from over-interpreting
this result. Future research should elucidate the reasons why IAT measures fail to show
predictive validity even if the IATs use the same stimuli as direct self-reports that
significantly predict behaviour. Valence influences, which we consider in the
Discussion, are one possible factor that may confound IAT effects and that may
represent a threat to the validity of IAT measures.
Discussion
This study explored the psychometric properties of an anxiousness and an angriness
IAT. The sequence of the IATs was counterbalanced and their predictive validity for
anxious versus angry behaviour after emotion inductions was examined.
The anxiousness IAT correlated with direct anxiousness measures and added
incremental validity over direct measures for the prediction of anxious behaviour.
The angriness IAT neither correlated with direct angriness measures nor did it show
predictive validity for angry behaviour. Additionally, there was an unexpected order
effect on the correlation between the IATs. In the next sections, we discuss order effects
on IAT correlations and explore possible reasons why the angriness IAT failed to show
predictive validity.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
389
Order effects on IAT correlations
In this study, we counterbalanced the order of an anxiousness and an angriness IAT and
found that both tests were, like the direct anxiousness and angriness measures, not even
marginally correlated when the anxiousness IAT was completed after the angriness IAT.
In contrast, the IATs were significantly correlated when the anxiousness IAT was
completed before the angriness IAT. The difference between the two correlations was
marginally significant and was attributed to the transfer of a positive-negative selfdimension from the anxiousness IAT to the angriness IAT. Valence ratings of the IAT
stimuli corroborated the hypothesis that a positive-negative dimension was more salient
in the anxiousness IAT than in the angriness IAT. Recently, the order effect on the
correlations between the IATs was replicated in an independent study (Teige et al.,
2004).
The account of the order effect in terms of a positive-negative task recoding is
somewhat related to the salience asymmetries account that was recently used to explain
IAT effects (Rothermund & Wentura, 2004; but see also, Greenwald, Nosek, Banaji, &
Klauer, 2005). According to the salience asymmetries account, participants base their
categorization of the bipolar IAT concepts primarily on the salient category (the ‘figure’)
and neglect the non-salient category (the ‘ground’). The salience asymmetries account
also proposes that negative IAT categories are more salient than positive ones
(Rothermund & Wentura, 2004). At least for the direct valence ratings, only the positive
but not the negative categories of the anxiousness and the angriness IAT were rated
differently. Consequently, the salience asymmetries account cannot explain why the
transfer effect was asymmetrical (from the anxiousness IAT to the angriness IAT, but not
vice versa) because the salience asymmetries account refers to the negative category,
and the valence-ratings of the anxiousness and the angriness IAT stimuli differed only
with respect to the positive category. Importantly, the salience asymmetries account
attempts to explain IAT effects for particular IATs and does not explain context or
transfer effects in multiple assessment. We assume that both IAT categories (e.g. ‘selfconfident’ and ‘anxious’) contribute to IAT effects and that concept discriminations
within the IAT are based on the valence or semantic contrasts between these concepts
(cf. Greenwald et al., 2005; Mierke & Klauer, 2003).
There are several other studies that employed more than one IAT within one sample
and counterbalanced the IAT presentation order (e.g. Greenwald et al., 2002, 2003;
Teachman, Gregg, & Woody, 2001; Teachman & Woody, 2003; Wiers, van Woerden,
Smulders, & de Jong, 2002). Many of these studies do not report order effects, or they
report complex interaction effects with order and other variables (e.g. gender or drinker
type, Wiers et al., 2002) on mean IAT effects. In many cases, sample sizes are too small
(ns , 30) to explore order effects on correlations, and none of these studies used more
than one self-concept IAT. The only study that used more than one self-concept IAT
(one anxiety and one extraversion IAT) is, to our knowledge, a study by Schmukle and
Egloff (2005) that does not report any effects of IAT order. At the present level of
knowledge, it seems reasonable to assume that there are order effects on IATs, especially
on the size of IAT effects (Nosek, Greenwald, & Banaji, in press) that may also affect the
assessment of multiple personality attributes. However, to what extent and why
presentation order affects the correlations of the IATs remains a question for future
research. Because order effects seem to be strongest from the first to the second IAT,
future research designs may include neutral control IATs (e.g. a geometrical objects IAT,
Mierke & Klauer, 2003) as the first IAT in order to reduce effects of presentation order
(Nosek et al., in press).
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
390
Konrad Schnabel et al.
Correlations of IAT measures may be influenced not only by the order of the IATs but
also by whether direct measures are presented before or after the IATs. In a recent metaanalysis, Hofmann, Gawronski, Gschwendner, Le, and Schmitt (2005) found no
consistent effects of the measurement order of direct and indirect measures on the
correlations between these measures. In order to sidestep possible transfer effects from
direct measures on the IATs we asked participants to complete direct self-ratings at
home within one week before the IAT assessments at the laboratory. Directly before the
IATs, participants completed a short concentration test (shortened version of the d2
test) as a justification of the cover story (‘personality and concentration’).
This concentration test took only 100 seconds to be completed. Nevertheless, it may
have produced subtle effects on the IAT scores. It is, therefore, important that the order
effect on the IAT correlations was replicated in another study that did not employ the d2
test and presented direct self-report measures before the IATs.
Semantic meaning and valence
Assuming that participants have recoded the IAT task in terms of a positive-negative selfevaluation, the present findings raise a question about the extent to which IAT measures
are driven by the semantic meaning as opposed to the positive or negative valence of the
stimuli. If IATs mainly reflect the ease with which one combines positive versus negative
stimuli with me, then the IATs represent self-esteem IATs (e.g. Greenwald & Farnham,
2000), rather than indirect measures of different personality attributes. Can the
empirical findings of different self-concept IATs be reinterpreted in terms of implicit
measures of self-esteem?
Concerning the anxiousness IAT, the answer might be ‘yes’. In the studies by Egloff
and Schmukle (2002), the anxiousness IAT predicted performance decrement due to
failures in a concentration test, and anxious behaviour during an evaluative speech task.
Both behaviours may also be predicted by a ‘pure’ self-esteem IAT that does not directly
refer to anxiousness (cf. Greenwald & Farnham, 2000). The results from the anxiousness
IAT of the present research may be reinterpreted using the same logic. The same
reasoning can be applied to the shyness IAT (Asendorpf et al., 2002) as well, such that
shy behaviour could be related to low self-esteem. Already at the level of direct
measures, shyness and anxiousness are negatively correlated with self-esteem (Cheek &
Melchior, 1990; Judge, Erez, Bono, & Thoresen, 2002). Thus, it is difficult to disentangle
valence and specific semantic meaning in anxiousness and shyness because a valid
portion of these attributes already contains negative self-evaluation.
Some studies have attempted to separate valence from semantic effects on IAT
measures. For instance, Rudman, Greenwald, and McGhee (2001) showed that the
positive or negative valence of stimuli affected the IAT in addition to the relevant
attribute dimension. In their studies, female participants showed no IAT effect in a
gender stereotype IAT when stereotypically female attributes (e.g. weak) were negative
and stereotypically male attributes (e.g. powerful) were positive. This was explained by
a tendency of the female participants not to combine their own gender with negatively
valenced attributes. Females showed the expected stereotypic gender associations in the
IAT only when the gender attributes were balanced for valence. Thus, the valence of the
gender attributes influenced the gender IAT even if participants did not categorize the
stimuli according to their positive or negative valence. On the other hand, results from
Steffens and Plewe (2001) show that the stereotypic gender association of pleasant (e.g.
empathic, gentleman-like) and unpleasant (e.g. bitchy, violent) attributes has an effect
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
391
on the gender attitude IAT. Female participants showed stronger associations between
female names and positive attributes and between male names and negative attributes
when the positive attributes were stereotypically female and the negative attributes
were stereotypically male. Thus, even if participants had to categorize the stimuli
according to their positive or negative valence, the semantic meaning of the attributes
(i.e. stereotypically male or female) influenced the IAT effect. Teachman et al. (2001)
employed a valence IAT (bad-good) and two fear-specific IATs (afraid-unafraid, dangersafety) to assess associations with spiders and snakes in a group of participants that were
either spider or snake phobic. Analyses of covariance showed that the fear-specific IATs
yielded different effects for spider- versus snake-phobic participants, even after
controlling for the impact of the valence IAT. In summary, IATs are probably affected by
both specific semantic and evaluative information and context effects seem to play an
important role; for example, whether the semantic or evaluative dimension of the
attributes is made salient. Although it may be difficult to separate semantic meaning and
valence because valence information is an important part of the semantic information,
future research should examine to which extent the IAT is influenced by the positive
and negative valence or by the specific semantic meaning of the stimuli. One possibility
to circumvent the problem of valence confoundings is to balance the IAT attribute
categories for valence.
Behaviour prediction through direct and indirect measures
The observer judgments of anxious behaviour during the speech and the observer
judgments of angry behaviour during the computer crash were predicted by the direct
anxiousness and angriness self-ratings, respectively, that participants completed at home
1 week before the laboratory experiments. Additionally, the anxiousness IAT added
incremental validity over direct measures for the prediction of the observer anxiety
judgment thereby replicating results from Egloff and Schmukle’s (2002) Study 4. In
contrast to Egloff and Schmukle’s results, the observer anxiety judgments were also
predicted by the direct measures in our study. This might be due to the fact that we
included several direct measures in addition to the trait form of the STAI. The situationspecific direct measures (i.e. the emotionality subscale of the Speaking Anxiety
Questionnaire and the bipolar state anxiety items) were particularly strong predictors for
the observer anxiety judgment in the present study. Yet, the trait form of the STAI also
correlated marginally with the observer anxiety judgment, r ¼ .19, p , .10, whereas this
was not true for Egloff and Schmukle’s study, r ¼ .12, ns. However, this correlation
difference was only small, and the lack of predictive validity of the direct anxiousness
measure in Egloff and Schmukle’s Study 4 might also be attributed to the small sample size
(N ¼ 33). Thus, the present study is in line with the expectation that direct measures
show small to moderate validity for the prediction of behaviour (Funder, 1999).
One could argue that the anxiousness IAT showed incremental validity only because
it was completed within the laboratory experiment that also explored the anxious
behaviour, whereas the direct anxiousness measures were completed at home.
Therefore, the anxiousness IAT might capture situational or occasion-specific effects
that are not reflected by the direct anxiousness measures. Importantly, the anxiousness
IAT explained unique portions of variance even if the direct state anxiety measure was
included into the regression analysis. Thus, it seems reasonable that the anxiousness IAT
assesses aspects of the personality self-concept that are valid for the prediction of
behaviour but that are not captured by direct measures.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
392
Konrad Schnabel et al.
Compared with the anxiousness IAT, the angriness IAT did not add incremental
validity over direct measures for the prediction of behaviour and did not even correlate
with the observer anger judgment. We explored whether this lack of predictive validity
was moderated by the relatively small anger induction effect. Contrary to expectations,
the IAT-behaviour correlation was even lower for participants that reported high state
anger. Additional analyses showed that the results were the same if the state anger raw
scores (rather than the residuals of a regression on baseline anger) or the difference
measure of the induction effect (state anger minus baseline anger) were induced as
moderator variables. In contrast, the correlation between direct angriness measures and
the observer anger judgment was not moderated by self-reported state anger. We do not
want to over-interpret this result, and most probably, it is simply one indicator of the
lack of validity of the angriness IAT.
Another problem of the anger induction was that some participants became
suspicious about the computer crash as being part of the experiment. In contrast to
anxiety, it is hard to induce anger in participants if they realize that it was intended to
provoke their anger. We pre-tested another paradigm to induce anger (i.e. the hot sauce
paradigm; cf. Harmon-Jones, & Amodio, in press) and found the computer crash to be
both more suitable for the observation of angry behaviour and less transparent for
participants. On the other hand, it is possible that, although they were mixed with
conscientiousness items, the state measures for anxiety and anger made the participants
more aware of the experimental manipulations. We wanted to include the state
measures because of two reasons. First, we wanted to check whether the emotion
inductions were successful at least at the explicit level. Second, we wanted to explore
the incremental validity of the IAT measures over both direct trait and state measures.
The good predictive validity of the direct anxiousness and angriness measures and of the
anxiousness IAT demonstrates that the anxiety and anger induction were successful
despite the presentation of the direct state measures.
Because we conceptualized the anxiousness and the angriness IAT as trait measures,
we acknowledge that it would have been desirable to validate them with more than one
situation. If we aggregated anxious and angry behaviour across multiple situations, then
we certainly would have obtained higher correlations at least with the anxiousness IAT.
However, this would have been extremely difficult because it requires that participants
comply with similarly long laboratory sessions on multiple days (to minimize transfer
effects from one situation to another). However, our 1-day approach did not fare too
badly as was indicated by the correlations between the direct anxiousness measures and
anxious behaviour during the speech, and between the direct angriness measures and
angry behaviour during the computer crash.
Angriness, agreeableness, anger expression and approach behaviour
This study explored implicit and explicit representations of the personality self-concept
of anxiousness and angriness. Explicit representations were assessed with bipolar
anxiousness and angriness self-ratings. Implicit representations were assessed by using
the same words as stimuli within the IATs. The convergent validity of the bipolar
anxiousness self-ratings with widespread anxiousness scales was high, r . .70.
In contrast, the correlation between the bipolar angriness self-ratings and the trait
form of the STAXI was only moderate, r ¼ .45. This might be due to the
conceptualization of anxiousness and angriness in the present study as orthogonal
factors within the Big Five model of personality.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
393
Conceptually and empirically, anxiousness versus self-confidence was strongly
related to neuroticism, and unrelated to agreeableness. Angriness versus self-control was
weakly related to neuroticism, and strongly related to agreeableness. In contrast, the
trait form of the STAXI is intermediately related with emotional instability or neuroticism
(Spielberger, 1988), and was also significantly correlated with all direct anxiousness
measures in the present study. In contrast with the trait form of the STAXI, the present
conceptualization of angriness refers more to agreeableness and less to emotional
instability or neuroticism. This may account for the moderate correlation between the
bipolar angriness self-ratings and the trait form of the STAXI. Nevertheless, the scale was
labelled angriness because it is less broad than the Big Five dimension of agreeableness.
Alternatively, angriness versus self-control may be considered as a combination of
high anger-out and low anger-control, which are strongly negatively correlated.
Moreover, anger-out and anger-control show the same intermediate correlations with
the trait form of the STAXI as the bipolar angriness self-ratings (Schwenkmezger et al.,
1992). Thus, the bipolar angriness self-ratings may more directly refer to styles of anger
expression than the trait form of the STAXI. A more direct relation to angry behaviour
within the bipolar angriness self-ratings is also suggested by the somewhat higher
correlations with the observer anger judgment than those obtained for the trait form of
the STAXI (see Table 3).
Anger is a negative emotion that is related to approach behaviour (e.g. Lazarus,
1991). In contrast, anxiety is related to avoidance behaviour that is true for most of the
negative emotions (e.g. sadness, disgust). Owing to the relation of state anger to
approach motivation, anger is associated with different EEG activation than anxiety
(Harmon-Jones & Sigelman, 2001). Possibly, the automatic categorization of stimuli
within the angriness IAT was somehow obstructed because angry versus self-control
combines approach-related words (e.g. angry) with negative valence, and avoidancerelated words (e.g. self-control) with positive valence. In contrast, avoidance-related
words (e.g. anxious) are combined with negative valence, and approach-related words
(e.g. self-confident) with positive valence in the anxiousness IAT. Generally, positive
valence is more strongly associated with approach motivation, whereas negative
valence is more strongly associated with avoidance motivation (e.g. Neumann,
Förster, & Strack, 2003). However, within the angriness IAT, motivational direction and
valence of the stimuli are inversely related. This might distort the automatic
categorization of angry versus self-controlled, and further accounts for (a) the lower
internal consistency within the angriness IAT (.66) than within the anxiousness IAT
(.72), (b) the lack of convergent validity of the angriness IAT, and (c) the susceptibility of
the angriness IAT to the transfer effect from the anxiousness IAT.
Conclusion
The present study replicated findings from Egloff and Schmukle (2002) by showing that
an anxiousness IAT added incremental validity over and above direct measures for the
prediction of anxious behaviour. These results illustrate that the IAT is able to assess
inter-individual differences that are valid for the prediction of behaviour but that are not
accessible with direct measurement procedures. On the other hand, the IAT’s potential
for the assessment of multiple personality dimensions seems to be restricted.
The angriness IAT showed somewhat unsatisfactory internal consistency and seemed to
be affected by the transfer of a positive-negative self-dimension from the anxiousness
IAT. Importantly, this transfer effect was recently replicated (Teige et al., 2004). Future
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
394
Konrad Schnabel et al.
research should explore the circumstances under which the IAT is affected by the
valence or by the specific semantic meaning of the stimuli and to what extent. As long as
these questions remain unsolved, it is unclear whether self-concept IATs assess interindividual differences in self-representations over and above self-evaluation. The exciting
potential offered by the IAT is that this measurement procedure assesses valid implicit
self-representations that are different from, and not accessible with, conventional selfreport measures (Asendorpf et al., 2002; Egloff & Schmukle, 2002). However, the IAT’s
susceptibility to context effects, the unresolved issue of effects of valence or specific
semantic meaning, and the partially low reliability show that the IAT is not ready to be
used as a standard procedure for individual diagnosis.
Acknowledgements
This research was supported by a grant of the German Research Foundation to J. B. Asendorpf &
R. Banse (As 59/9-1). We wish to thank Jekatarina Cechini, Dennis Mocigemba, and Ulrike Schild
for their help as experimenters, and Boris Egloff and Monika Wiedig for their advice on developing
the emotion inductions and for their kind help in providing anchor stimuli for our emotion
judgments.
References
Asendorpf, J. B., Banse, R., & Mücke, D. (2002). Double dissociation between implicit and explicit
personality self-concept: The case of shy behavior. Journal of Personality and Social
Psychology, 83, 380–393.
Bargh, J. A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait
construct and stereotype activation on action. Journal of Personality and Social Psychology,
71, 230–244.
Behringer, J. (1994). Experimental run time system (Version 3.18). Frankfurt, Germany: BeriSoft
Corporation.
Brickenkamp, R. (1994). Test d2 Aufmerksamkeits-Belastungs-Test (8th ed.) [Test d2 attention
stress test]. Göttingen: Hogrefe.
Cheek, J. M., & Melchior, L. A. (1990). Shyness, self-esteem, and self-consciousness. In
H. Leitenberg (Ed.), Handbook of social and evaluation anxiety (pp. 47–82). New York:
Plenum Press.
Costa, P. T. J., & McCrae, R. R. (1992). Revised NEO Personality inventory and NEO five factor
inventory professional manual. Odessa, FL: Psychological Assessment Resources.
Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of
psychopathology. Journal of Consulting Psychology, 24, 349–354.
De Houwer, J. (2001). A structural and process analysis of the implicit association test. Journal of
Experimental Social Psychology, 37, 443–451.
De Houwer, J. (2002). The implicit association test as a tool for studying dysfunctional associations
in psychopathology: Strengths and limitations. Journal of Behavior Therapy and
Experimental Psychiatry, 33, 115–133.
Egloff, B., & Schmukle, S. C. (2002). Predictive validity of an implicit association test for assessing
anxiety. Journal of Personality and Social Psychology, 83, 1441–1455.
Egloff, B., & Schmukle, S. C. (2003). Does social desirability moderate the relationship between
implicit and explicit anxiety measures? Personality and Individual Differences, 35,
1697–1706.
Egloff, B., Schmukle, S. C., Burns, L. R., & Schwerdtfeger, A. (in press). Spontaneous emotion
regulation during evaluated speaking tasks: Associations with negative affect, anxiety
expression, memory, and physiological responding. Emotion.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
Anxiousness and angriness
395
Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition research: Their meaning
and use. Annual Review of Psychology, 54, 297–327.
Funder, D. C. (1999). Personality judgment: A realistic approach to person perception. San Diego,
CA: Academic Press.
Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem, and
stereotypes. Psychological Review, 102, 4–27.
Greenwald, A. G., Banaji, M. R., Rudman, L. A., Farnham, S. D., Nosek, B. A., & Mellot, D. S. (2002).
A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept. Psychological
Review, 109, 3–25.
Greenwald, A. G., & Farnham, S. D. (2000). Using the implicit association test to measure selfesteem and self-concept. Journal of Personality and Social Psychology, 79, 1022–1038.
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in
implicit cognition: The implicit association test. Journal of Personality and Social
Psychology, 74, 1464–1480.
Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the implicit
association test: 1. An improved scoring algorithm. Journal of Personality and Social
Psychology, 85, 197–216.
Greenwald, A. G., Nosek, B. A., Banaji, M. R. & Klauer, K. C. (2005). Validity of the salience
asymmetry interpretation of the IAT: Comment on Rothermund and Wentura (2004). Journal
of Experimental Psychology: General, 134, 420–425.
Harmon-Jones, E., & Amodio, D. M. (in press). Social psychological strategies for emotion
induction. In J. J. B. Allen & J. A. Coan (Eds.), The handbook of emotion elicitation and
assessment. New York: Oxford University Press.
Harmon-Jones, E., & Sigelman, J. (2001). State anger and prefrontal brain activity: Evidence that
insult-related relative left-prefrontal activation is associated with experienced anger and
aggression. Journal of Personality and Social Psychology, 80, 797–803.
Hodapp, V. (1991). Das Prüfungsängstlichkeitsinventar TAI-G: Eine erweiterte und modifizierte
Version mit vier Komponenten [The Test Anxiety Inventory TAI-G: An expanded and modified
version with four components]. Zeitschrift für Pädagogische Psychologie, 5, 121–130.
Hofmann, W., Gawronski, B., Gschwendner, T., Le, H., & Schmitt, M. (2005). A meta-analysis on the
correlation between the Implicit Association Test and explicit self-report measures.
Personality and Social Psychology Bulletin, 31, 1369–1385.
Judge, T. A., Erez, A., Bono, J. E., & Thoresen, C. J. (2002). Are measures of self-esteem,
neuroticism, locus of control, and generalized self-efficacy indicators of a common core
construct? Journal of Personality and Social Psychology, 83, 693–710.
Krohne, H. W., Egloff, B., Kohlmann, C. W., & Tausch, A. (1996). Untersuchungen mit einer
deutschen version der positive and negative affect schedule (PANAS) [Investigations with a
German version of the positive and negative affect schedule (PANAS)]. Diagnostica, 42,
139–156.
Laux, L., Glanzmann, P., Schaffner, P., & Spielberger, C. D. (1981). Das state-trait-angstinventar
[The state-trait anxiety inventory]. Weinheim: Beltz.
Lazarus, R. S. (1991). Emotion and adaptation. New York: Oxford University Press.
Lück, H. E., & Timaeus, E. (1969). Skalen zur Messung Manifester Angst (MAS) und sozialer
Wünschbarkeit (SDS-E und SDS-CM) [Scales for the assessment of manifest anxiety (MAS) and
social desirability (SDS-E and SDS-CM)]. Diagnostica, 15, 134–141.
Mierke, J., & Klauer, K. C. (2003). Method-specific variance in the implicit association test. Journal
of Personality and Social Psychology, 85, 1180–1192.
Neumann, R., Förster, J., & Strack, F. (2003). Motor compatibility: The bidirectional link between
behavior and evaluation. In J. Musch & K. C. Klauer (Eds.), The psychology of evaluation.
Affective processes in cognition and emotion ( pp. 762–768). Mahwah, NJ: Erlbaum.
Nosek, B. A., Greenwald, A. G. & Banaji, M. R. (in press). The Implicit Association Test at age 7:
A methodological and conceptual review. In J. A. Bargh (Ed.), Automatic Processes in Social
Thinking and Behavior. New York: Psychology Press.
Copyright © The British Psychological Society
Reproduction in any form (including the internet) is prohibited without prior permission from the Society
396
Konrad Schnabel et al.
Ostendorf, F. (1990). Sprache und Persönlichkeitsstruktur: Zur Validität des Fünf-FaktorenModells der Persönlichkeit [Language and personality structure: Validity of the five-factor
model of personality]. Regensburg, Germany: S. Roderer.
Rothermund, K., & Wentura, D. (2004). Underlying processes in the implicit association test (IAT):
Dissociating salience from associations. Journal of Experimental Psychology: General, 133,
139–165.
Rudman, L. A., Greenwald, A. G., & McGhee, D. E. (2001). Implicit self-concept and evaluative
implicit gender stereotypes: Self and ingroup share desirable traits. Journal of Personality and
Social Psychology, 27, 1164–1178.
Schmukle, S. C., & Egloff, B. (2005). A latent state-trait analysis of implicit and explicit personality
measures. European Journal of Psychological Assessment, 21, 100–107.
Schwenkmezger, P., Hodapp, V., & Spielberger, C. D. (1992). Das State-Trait-ÄrgerausdrucksInventar STAXI [The State-Trait-Anger-Expression-Inventory STAXI]. Göttingen: Huber.
Spielberger, C. D. (1980). Test anxiety inventory (Test Attitude Inventory). Preliminary
professional manual. Palo Alto, CA: Consulting Psychologists Press.
Spielberger, C. D. (1988). State-trait-anger-expression-inventory (STAXI). Research edition.
Odessa, FL: Psychological Assessment Resources.
Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). STAI. Manual for the state-trait anxiety
inventory. Palo Alto, CA: Consulting Psychologists Press.
Spitznagel, A., Schlutt, S., & Schmidt-Atzert, L. (2000). Redängstlichkeit: Entwicklung eines
mehrdimensionalen Verfahrens [Public speaking anxiety: Construction of a multidimensional
inventory]. Diagnostica, 46, 47–55.
Steffens, M. C., & Plewe, I. (2001). Items’ cross-category associations as a confounding factor in
the implicit association test. Zeitschrift für Experimentelle Psychologie, 48, 123–134.
Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin,
87, 245–251.
Stemmler, G., Heldmann, M., Pauls, C. A., & Scherer, T. (2001). Constraints for emotion specificity
in fear and anger: The context counts. Psychophysiology, 38, 275–291.
Stöber, J. (1999). Die Soziale-Erwünschtheits-Skala-17 (SES-17): Entwicklung und erste Befunde zu
Reliabilität und Validität. Diagnostica, 45, 173–177.
Strack, F., & Deutsch, R. (2004). Reflective and impulsive determinants of social behavior.
Personality and Social Psychology Review, 8, 220–247.
Taylor, J. A. (1953). A personality scale of manifest anxiety. Journal of abnormal social
Psychology, 48, 285–290.
Teachman, B., Gregg, A., & Woody, S. (2001). Implicit processing of fear-relevant stimuli among
individuals with snake and spider fears. Journal of Abnormal Psychology, 110, 226–235.
Teachman, B., & Woody, S. (2003). Automatic processing among individuals with spider phobia:
Change in implicit fear associations following treatment. Journal of Abnormal Psychology,
112, 100–109.
Teige, S., Schnabel, K., Banse, R., & Asendorpf, J. B. (2004). Assessment of multiple implicit selfconcept dimensions using the extrinsic affective Simon task (EAST). European Journal of
Personality, 18, 495–520.
Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of
positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology,
54, 1063–1070.
Wiedig, M. (2004). Ärger und soziale Regeln: Ärgererleben und Ärgerverhalten in der Selbst- und
Fremdwahrnehmung [Anger and social rules: Anger experience and anger behavior in selfratings and observer ratings]. Hamburg: Kovac.
Wiers, R. W., van Woerden, N., Smulders, F. T., & de Jong, P. J. (2002). Implicit and explicit alcoholrelated cognitions in heavy and light drinkers. Journal of Abnormal Psychology, 111,
648–658.
Received 15 November 2004; revised version received 16 March 2005