Running head: MEASUREMENT OF NEGATIVITY BIAS
Measurement of Negativity Bias in Personal Narratives Using Corpus-Based
Emotion Dictionaries
Shuki J. Cohen
New York University, Department of Psychology
Corresponding Author:
Shuki J. Cohen
Psychology Department
John Jay College of Criminal Justice
445 W 59th St rm# 2402
New York, NY 10019
Ph: (646) 557-4627 Fax: (212) 237-8930
e-mail: shcohen@jjay.cuny.edu
Yale University School of Medicine
Department of Psychiatry
34 Park St. Suite #B-38
New Haven, CT 06519
1
Measurement of Negativity Bias 2
Abstract
This study presents a novel methodology for the measurement of negativity bias using positive
and negative dictionaries of emotion words applied to autobiographical narratives. At odds with
the cognitive theory of mood dysregulation, previous text-analytical studies have failed to find
significant correlation between emotion dictionaries and negative affectivity or dysphoria. In the
present study, an a priori list dictionary of emotion words was refined based on the actual use of
these words in personal narratives collected from close to 500 college students. Half of the
corpus was used to construct, via concordance analysis, the grammatical structures associated
with the words in their emotional sense. The second half of the corpus served as a validation
corpus. The resulting dictionary ignores words that are not used in their intended emotional
sense, including negated emotions, homophones, frozen idioms etc. Correlations of the resulting
corpus-based negative and positive emotion dictionaries with self-report measures of negative
affectivity were in the expected direction, and were statistically significant, with medium effect
size. The potential use of these dictionaries as implicit measures of negativity bias and in the
analysis of psychotherapy transcripts is discussed.
Keywords: Text-Analysis, Dysphoria, Negativity Bias, Personal Narratives, Corpus Linguistics
Footnotes: The author wishes to thank Patrick E. Shrout and the late Carol Feldman and
Joan Welkowitz for their advice in conducting this research. Also, many thanks to the numerous
research assistants who assisted with various parts of this research, especially to: Julia Betensky,
Venessa Bikhazi ,Candis Conover, Heather Holahan, Melanie Fox, Tyson Fur, Jonathan
Kirschner, Jen Leeder, Julie McCarroll, Joanna Murstein, Jonathan Shaffer and Sam Zun.
Measurement of Negativity Bias 3
Measurement of Negativity Bias in Personal Narratives Using Corpus-Based Emotion
Dictionaries
Introduction
Bias in the processing of emotional information has been widely established as one of the
most fundamental features of psychological disorders as of individual personality. Extensive
research has repeatedly demonstrated that people high on neuroticism or negative affectivity
selectively perceive, attend to, interpret and recall negatively charged or ambiguous events more
negatively than normal controls (Watson & Clark, 1984; Clark et al., 1999; Rusting, 1999;
Abramson et al., 1989; Ingram, 1984; Rector et al., 1998, Rude et al., 2003). In cognitive
psychology, as well as in its psychotherapeutic applications, the cognitive overrepresentation of
negative information is conceptualized within the framework of depressive self-schema (e.g.
Beck, 1976; Markus, 1977; Segal & Ingram, 1994; Nasby & Kihlstrom, 1986). Evidence for the
existence of depressogenic self-schema among both dysphoric subjects as well as clinically
depressed patients has been demonstrated in experiments involving attention, recall, lexical
decision, mood priming, and other cognitive capacities (for reviews see Isen, 1984; Mathews &
MacLeod, 1994; Rector et al., 1998; Rusting, 1998; Scher et al., 2004).
Negativity bias in the self-schema has been mostly studied using self-report
questionnaires (Shaw & Dobson, 1981; Dobson & Breiter, 1983; Oliver & Baumgart, 1985;
Beck et al., 1988; Clark, 1988; Haaga et al., 1991; Ingram et al., 1995; Chien & Dunner, 1996;
Glass & Arnoff, 1997; Alloy et al., 1999; Winters et al., 2002). Self-reports have mostly
demonstrated excellent psychometric properties and are straightforward, practical and
inexpensive to administer. Further, self-reports have demonstrated theory-consistent
Measurement of Negativity Bias 4
correspondence with cognitive content and schematic organization of emotional material.
However, self-report questionnaires are also frequently criticized due to their susceptibility to the
introspective limits of the respondent and to response bias, including impression management
and self-deception or defensiveness (e.g. Winters & Neale, 1985; Polaino & Senra, 1991;
Shedler et al., 1993; Gotlib et al., 1995; Lindeman & Verkasalo, 1995; Paulhus & John, 1998;
Stone et al., 2000; Wenzlaff & Wegner, 2000; Rude et al., 2001; Nosek, 2005). Moreover,
cognitive theory regards the depressogenic schema as a stable cognitive style that is minimally
affected by states of mood or depressive symptomatology, but numerous self-reports have failed
to detect negativity bias in remitted patients under normal mood conditions (e.g. Persons &
Miranda, 1992; Segal & Ingram, 1994; Haeffel et al., 2005; Fresco et al., 2006).
One vehicle that was suggested to circumvent the high face validity of self-reports is the
analysis of verbal behavior. Studies that have used autobiographic speech samples to assess the
negative emotional bias of the speaker rest on the premise that a) the ability of the speaker to
control word choice is limited due to the rapidity of fluent speech and b) that speech may afford
an almost instantaneous account of the cognitive content of the mind as it reaches awareness,
thus making it the fastest method of introspection. Negativity bias in the emotional content of
spontaneous speech has shown consistent associations with indices of mental health. For
example, Gottschalk and Gleser (1969 p. 106) found significant correlations between their
measure of verbal anxiety and the emotional instability subscale of the 16PF, and Gottschalk
(1997) found significant correlations between the same scale and self-reported anxiety. Other
studies that used manual-based content analysis of spontaneous speech have also reported
significant correlations with the cognitive negativity bias of the speaker (e.g. Viney, 1983;
Measurement of Negativity Bias 5
Watkins & Rush, 1983; Peterson, 1992; Cacioppo et al., 1997; Davison et al., 1997; Hurlburt,
1997; Hartman-Hall & Haaga, 1999)
Manual-based content analysis of spontaneous speech has been criticized for being timeconsuming, training-intensive and susceptible to rater drift and other rater-based biases (e.g.
Hurlburt, 1997; Pennebaker et al., 2003). In contrast, computerized text-analysis can process
large amounts of transcribed speech in considerably shorter time (Gottschalk & Bechtel, 1982;
Pennebaker et al., 2003; Alpers et al., 2005). Moreover, several studies that used computerized
text-analysis of multiple categories of words have demonstrated superiority of this technique
over human raters in assessing mental health (Rosenberg et al., 1990; Pennebaker et al., 1997;
Pennebaker & King, 1999).
This study reports the construction and initial validation of an index of negativity bias
based on context-based dictionaries for positive and negative emotional language. These
dictionaries can be plugged into text-analytical software to obtain a metric representing the
positive and negative emotional content of the speech sample. The construction process is
described briefly, while a more detailed account is now under review. The advantage of these
dictionaries, as well as the importance of contextualized dictionary entries for the computerized
detection of emotional language are then discussed.
Method
General Overview:
The construction of both positive and negative dictionaries followed several phases. In
Phase 1, three research assistants independently detected candidates for emotion words in
Measurement of Negativity Bias 6
transcripts of 240 narratives obtained from undergraduate students using standard elicitation
technique. The resulting list was consolidated and then contrasted and completed using other
text-analytical dictionaries In Phase 2, the context in which each of the candidate emotion words
occurred in the narrative was examined using concordances specific to each word. This phase
detected word combinations that are reliably associated with the use of the word in its intended
emotional meaning. Conversely, this phase also detected idioms or word combinations in which
the candidate words were not used in their emotional sense. To avoid the ‘dilution’ of the
proposed dictionary, this list was subtracted from the final metric. Overall, each positive or
negative dictionary is comprised of a list of words, word stems or idioms that constitute the
emotionality metric, and an ‘anti-dictionary’ of word combinations that should be subtracted
from the dictionary to ensure its purported emotional meaning. Finally, to test the concurrent
validity of the proposed dictionaries, their association with two common self-report measures of
psychological distress and depression were examined. The Global Severity of symptom Index
(GSI) of the Symptom Checklist-90 (SCL-90) was used as a measure of general psychological
distress, and the Beck Depression Inventory (BDI) was used as a measure of dysphoria..
Participants:
Participants were 483 undergraduate students (345 female and 138 male) from a large
private urban northeastern university who received partial course credit for their participation.
All participants were self-identified as native speakers of North American English. The average
age of the subjects was 19.8 years.
Measurement of Negativity Bias 7
The participants were recorded telling a story concerning a recent disagreement they had
with a significant other, and then filled out a battery of self-report questionnaires covering
various aspects of their mental well-being and symptoms.
Personal narratives:
For the elicitation of speech samples, research assistants approached the participant with
the following request:
“I would like for you to tell me in your own words the details of a recent disagreement you had
with somebody who is close to you emotionally. It may be a small or big disagreement, as long
as you feel comfortable sharing it and it is with somebody who is or was close to you at the time.
You have five minutes to tell your story, and it will be recorded on the machine in front of you.
Five minutes is a long time for a story, so you may want to go into details. Also, I will not be
able to help you along or lead you with questions. I will stop the machine when the five minutes
are up. Do you have any questions? (subjects questions were answered at this point as succinctly
as possible, although almost invariably the subject replied negatively to this question). Ok, let me
know when I can start recording” (subjects to the experimenter when they were ready to
proceed).
The elicitation requests were delivered verbatim to avoid variability in the demand
characteristics of the task. The lack of verbal interaction with the subject was also designed to
standardize the task and preserve the unique narrative structure of each speaker. The stories were
Measurement of Negativity Bias 8
then transcribed according to a common standard in the field (Mergenthaler & Stinson, 1992),
using a 3-stage process to ensure quality control.
Computerized Text-analysis:
The text-analysis software was written by the author, and was comprised of a set of
computerized routines, or modules – each allowing the manipulation and analysis of the text in a
manner tailored to the task at hand. Like existing text-analytical packages, the software couls
search for whole words, word combinations, stems of words and word suffixes. The software
could also construct a concordance showing the immediate context of a given word, and
presented the results as both counts and frequency of the searched items (For a solid linguistics
and methodological background, including previous studies utilizing this technique see Sinclair,
1991). The software was designed as a programmer’s tool, rather than a user-ready package, to
ensure its accuracy and avoid the blind suppression of error messages – a common practice in
commercial text-analytical packages.
The Development of the Dictionary of Positive and Negative Emotional Language:
Phase 1:
A group of 3 research assistants-- all native speakers of US English-- inspected
independently the narratives of the first 240 subjects for positive and negative emotion language.
The definition of emotion used in this stage was broad, to foster maximal inclusion. The words
could include behaviors associated with emotions (e.g. yelling, crying, fighting, break* up, slam
Measurement of Negativity Bias 9
(door), hang up (phone) etc), emotional states (e.g. happy, upset, awkward, surprised, and
cognitive words frequently associated with emotional states (e.g. discouraging, appreciate, agree,
understand etc.)
Words, word stems or word combinations that were selected by two or more research
assistants were retained. This list was then merged with a comprehensive list of emotion words
from other, mostly text-analytical, dictionaries (Zuckerman et al., 1965; Stone et al., 1966;
Anderson, 1968; Dahl & Stengel, 1978; Sweeney & Whissell, 1984; Whissell et al., 1986;
Marchitelli, 1983; Marchitelli & Levenson, 1992; Pennebaker et al., 2001). The resulting masterlist comprised of 1,613 positive words and 1,927 negative words. After eliminating duplicate
words, words that have not been uttered by at least two speakers in the training corpus
(corresponding to frequency of below 1/10,000 words), and valenced words that connote
emotions but don’t represent them (e.g. noise, anniversary etc.), the final list of word candidates
contained 211 negative words and 161 positive words.
Phase 2:
In this phase, a concordance was constructed for each word in the list described in the previous
phase. The concordance presented the rater with the context in which the word appeared,
approximately two sentences before and after the word was uttered in the training corpus. Based
on this concordance a set of exclusion or inclusion criteria was crafted for the use of each word
in its intended emotional sense. In cases where no consistent grammatical structures involving
the word could be established, and no inclusion or exclusion criteria could be discerned, the
word was deemed ambiguous in its emotional sense and was eliminated from the dictionary. For
Measurement of Negativity Bias 10
example, whereas words like pretty, like or kind, were included in the positive emotion
dictionary of Phase 1, Phase 2 showed that their usage is too variable and is rarely consistently
emotional to merit inclusion. Almost invariably, the word pretty was used as an intensifier (e.g.
“pretty big”), the word like was used mostly as a conversational filler (e.g. “and I was, like,
totally shocked”), and the word kind was used as a qualifier or a hedge (e.g. “it was kind of
surprising to see him there”). The concordance for these words showed that they were used to
convey emotional meaning in less than 2% of the cases. Acknowledging the vast ‘dilution’ that
could result from including these words in the positive emotional dictionary, they were deleted at
this stage. However, conjugations of these words that reliably and consistently denoted
emotional meaning (e.g. liked, prettier etc.) were retained. Similarly, negations of emotion words
were added to the opposite-valence dictionary. For example, out of the 47 occurrences of the
word happy in the training corpus, 17 (36%) were negated, and of the 16 occurrences of the word
bother in the training corpus 11 (69%) were negated (e.g. “it didn’t bother me”). Including these
words in isolation could have resulted in considerable ‘noise’ in assessing the emotionality of the
narrative.
This phase represents a methodological hybrid between traditional computerized textanalysis and the more nuanced corpus-based concordance analysis. As such, the process
described above has the potential of being more sensitive to local context and word sense.
General Severity Index (GSI):
The Symptom Checklist -90- Revised (SCL-90-R; Derogatis et al., 1976; Derogatis &
Savitz, 2000) is one of the most commonly used and extensively studied instruments for the
Measurement of Negativity Bias 11
assessment of psychological distress. The SCL-90-R evaluates the degree of perceived emotional
distress caused by 90 common psychologically-relevant symptoms. Despite its theoretical
multidimensionality, the relatively low internal consistency of the scale has popularized the use
of its overall score, the Global Severity Index (GSI) as a measure of general psychological
distress (e.g. Cyr et al., 1985; Derogatis & Savitz, 2000).
Beck Depression Inventory (BDI):
The Beck Depression Inventory (BDI; Beck et al., 1961) is arguably the most commonlyused instrument for the assessment of dysphoria and screening for depression (e.g. Ritterband &
Spielberger, 1996). The subject is asked to rate the severity level of 21 common symptoms of
depression. The instrument has demonstrated adequate psychometric properties in numerous
settings and populations (for review see Beck et al., 1988). However, while the instrument was
designed as a measure of depression, a growing body of research has established its low
specificity, as evidenced by its moderate to high associations with measures of general
psychological symptomatology (e.g. Endler et al., 1998; Hill et al., 1986). This compromised
discriminant validity may be particularly pertinent for college population (see review in Gotlib,
1984).
Results
Split-half reliability of the emotional dictionaries
Measurement of Negativity Bias 12
Both Phase 1 and Phase 2 were based solely on the first 240 narratives. Therefore, it was
important to establish the split-half reliability of the corpus by comparing the emotional tone of
these narratives (i.e. the training corpus) with the rest of the stories. To accomplish that, the two
parts of the narrative corpus were compared using t-tests applied to the proportion of positive
and negative emotion words identified through the dictionaries. The proportion of emotion words
was defined as the number of emotion markers identified by the dictionary divided by the
number of words in the story. The training and the validation corpora did not differ in the
proportion of emotion words (Positive Emotions: t(239)=-0.59, p=0.555; Negative Emotions
t(239)=1.56, p=0.121, using 2-tail t-test under the heteroscedasticity assumption), thus
establishing the homogeneity of the corpus.
Concurrent Validity Checks for the Dictionary of Positive and Negative Emotional Language:
The association between the proportion of emotion words captured by the dictionaries
and self-report measures of dysphoria and psychological distress is shown in Table 1. Table 1
also shows the associations of dysphoria and distress with the proportion of emotion words as
captured by two leading text-analytical software packages – the Linguistic Inquiry and Word
Count (Pennebaker & Francis, 1996; Pennebaker et al., 2001) and the General Inquirer (Stone et
al., 1966).
For the General Inquirer scales, the positive and negative emotion dictionaries proposed
here were compared to the “negativ” and “positiv” scales from the Harvard-IV
Psychosociological Dictionary, which are comprised of 2,291 and 1,915 words of negative and
positive outlook, respectively. The corresponding dictionaries of the LIWC were the positive and
Measurement of Negativity Bias 13
negative emotion dictionaries. As Table 1 shows, the correlations between the corpus-based
emotion dictionaries and the GI and LIWC dictionaries are significantly different from zero, thus
conferring a modest degree of concurrent validity to the proposed corpus-based dictionaries.
However, the magnitude of the correlations ranges between small to moderate, with no
statistically significant correlation between the corpus-based positive emotion dictionary and the
LIWC positive emotion dictionary. In contrast, the corpus-based positive emotion dictionary
exhibits statistically significant correlation with the respective dictionary in the GI program.
However, the effect size of this association is small, and the dictionaries share only 6% of their
variance.
Construct Validity Checks for the Dictionary of Positive and Negative Emotional Language:
A crucial test for the construct validity of the proposed corpus-based dictionary is its
correlation with measures of psychological distress or dysphoria. As mentioned above, the
construct of psychological distress, with its related manifestations, namely depression or
dysphoria, is associated with preferential processing of negative information (for reviews see
Clark et al., 1999; Blaney, 1986; Segal & Ingram, 1994; Rusting, 1999; Rector et al., 1998;
Martin, 1985; Teasdale, 1999; Beck, 2002; Scher et al., 2004) has not been demonstrated reliably
in most previous computerized text-analytical studies.
Table 1 shows the correlations between the two measures of psychological distress (BDI
and GSI) and the proportion of positive and negative emotion words based on the corpus-based
dictionary. As an adjunct to the analysis of the concurrent validity of the corpus-based emotion
Measurement of Negativity Bias 14
dictionaries, Table 1 also shows the correlations of the proposed dictionaries with measures of
distress as compared with the two other text-analytical programs mentioned above.
As shown in Table 1, the corpus-based negative and positive emotions correlated
significantly, and in the expected direction, with measures of distress and dysphoria, while the
two other programs did not exhibit such a correlation. The exception to this general finding is the
LIWC’s negative emotion scale, which correlates significantly with the BDI. To assess the
statistical significance of the difference between the correlations of the self-reports with the 3
text-analytical systems, a procedure developed by Olkin & Finn (1990, 1995) for testing the
equality of dependent correlations was used. This technique was preferable to the common
methods of comparing correlation magnitude based on Fisher-test, since the correlations to be
compared were derived using the same sample, hence violating the independence assumption
inherent in the Fisher test. Table 2 the 95% Confidence intervals for the differences between the
correlations of the text-analytical scales and the outcome measures, as calculated by Olkin &
Finn (1990,1995) procedure. The analysis has shown that the difference between the correlations
is statistically significant. Therefore, the frequency of emotion words detected by the corpusbased dictionary was significantly more associated with measures of dysphoria and distress than
the two other text-analytical programs.
Discussion
This study examined a novel approach to the assessment of negativity bias, based on
computerized text-analysis of personal narratives that considers context information.
Incorporating concordance-based contextual information into the automatic identification of
Measurement of Negativity Bias 15
emotion words is an attractive alternative to extant text-analytical methodology, which currently
relies almost solely on decontextualized word lists. In lieu of amassing a large number of
emotion words or their synonyms, and tallying their occurrence in personal narratives, this study
showed that including simple usage-based rules to preferentially tally only words that are used in
their emotional sense robustly augmented the validity of the text-analytical dictionaries. The
construct validity for the proposed dictionaries was tested by examining their correlations with
common measures of general psychological distress (GSI) and dysphoria (BDI) on normal
undergraduate population. Both corpus-based positive and negative emotion dictionaries
demonstrated improved construct validity as compared to the two most studied text-analytical
programs, namely the LIWC and the General Inquirer.
Several methodological steps were taken to maximize the ecological validity of the
corpus-based emotional dictionaries. Firstly, the personal narratives in the corpus used to
construct the dictionaries concerned a recent disagreement with a significant other. This type of
narrative is highly frequent in psychotherapy, as well as general social settings (Horowitz, 1979;
Biber et al., 1998; McAdams, 2001; Bohanek et al., 2005). Further, Ample research has shown
that negative bias in processing emotional information is most consistently demonstrated when
the information pertains to the self or a significant other, but not as consistently when it concerns
generic person (Martin et al., 1983; Kuiper & Higgins, 1985; Bargh & Tota, 1988; Kuiper et al.,
1988; Mangan & Hookway, 1988; Baldwin, 1992; Cacioppo et al., 1997; Collins & Feeney,
2004). Secondly, the relatively passive role of the experimenter in the elicitation of the narrative
contributed to narratives that are similar in structure and content to those that are usually told to a
generic stranger. These narratives also look remarkably similar to those found in transcripts of
first sessions of psychodynamic psychotherapy or psychodiagnostic interviews. From the
Measurement of Negativity Bias 16
psycholinguistic and socio-linguistic point of view, this level of interaction minimize the effects
of familiarity, speech accommodation, politeness and other external influence on the subject’s
speech (e.g. Bell, 1984; Hudson, 1996; Shepard et al., 2001; Giles, 2001). Thirdly, the speakers’
unawareness of our interest in emotional language was crucial to the elicitation of emotional
narratives while minimizing the demand characteristics of the situation. To that aim, nowhere in
the description of the experiment to the subject was there any mention of emotions. Akin to a
clinical discourse, the request to talk about a disagreement with a significant other (rather than a
topic of the speaker’s choice) was designed to minimize simultaneously both the social
desirability demand to share with strangers harmlessly positive stories and the demand to
mention or elaborate on emotions per se. Further, avoiding a direct request to speak about
negative emotions arguably contributed to the validity of the corpus, as speakers could choose
the emotional elaboration level with which they felt most comfortable with a stranger. Indeed,
the corpus demonstrated a wide range of emotional elaboration levels, including a wide variety
of psycholinguistic devices to minimize emotionality: narrating the factual chain of events,
minimizing the emotional impact of the event, or even reframing it as a positive experience.
The results of this study underline the usefulness of theory-driven emotion dictionaries
for text-analysis of speech. The substantial correlations of the dictionaries with measures of
psychological distress make them a natural tool for exploring the link between psychopathology
and negative information processing bias using naturalistic speech as a medium. As mentioned
above, previous attempts to link text-analytical indices of negativity bias with emotional or
clinical presentation yielded mixed results. In general, theory-guided computerized textanalytical scales failed so far to demonstrate convincing correlations with either self-report
questionnaires or clinical interviews (Bohanek et al., 2005; Pennebaker & King, 1999; Williams
Measurement of Negativity Bias 17
et al., 2003; Stirman & Pennebaker, 2001), especially considering the robust correlations
between human ratings of speech emotionality and mental health indices (e.g. (Davison et al.,
1997; Lee & Peterson, 1997; Gottschalk & Gleser, 1969; Viney, 1983; Ruiz-Caballero &
Bermudez, 1995; Rusting, 1999; Klein et al., 1986; Cacioppo et al., 1997; Hurlburt, 1980;
Weintraub, 1989; Demorest et al., 1999; Luborsky & Crits-Christoph, 1990; Hermans, 1995;
Pennebaker et al., 2003). In contrast, those computerized text-analytical systems that have
achieved adequate construct validity when compared with clinically-validated measures are not
theoretically guided, but rather based on empirical, criterion-keyed or bottom-up principles
(Bucci & Freedman, 1981; Campbell & Pennebaker, 2003; Fertuck et al., 2004; Mergenthaler &
Bucci, 1999; Oxman et al., 1988; Pennebaker et al., 2003; Rosenberg et al., 1990; Rosenberg et
al., 1994; Segal et al., 1993; Spencer & Spencer, 1993; Stone, 1997). The dictionaries reported
here may be the only theory-based measures to demonstrate robust correlations with indices of
dysphoria and general psychological distress, comparable to those of human ratings of speech
emotionality.
The results of this study also underline the crucial contribution that contextual
information can make to the validity of text-analytical indices of emotionality. Unlike previous
text-analytical systems, this study identified contextual constraints on the text-analytical
dictionaries using concordance analysis of emotional markers that was applied to the corpus of
narratives (For an important previous attempt, albeit more limited and syntax-bound rather than
usage-based, to disambiguate emotional sense of words in computerized text-analysis see Kelly
& Stone, 1975). Concordance analysis revealed substantial sources of “noise” in the
decontextualized word-lists used currently in text-analytical programs. For example, while the
words pretty, like and kind are listed in most text-analysis dictionaries as positive emotion
Measurement of Negativity Bias 18
words, concordance analysis showed that over 97% of their occurrence is not consistent with
their intended positive emotional meaning. Rather, the word pretty is used as a quantifier, and
like and kind (especially as the phrase “kind of”, which constitutes about 1425/1435 or 99.3% of
all occurrences of kind in our sample), are used as fillers, filled pauses or hedges. Concordance
analysis also found that although speakers tend to prefer the affirmative sense of emotion words
rather than their negation, it is not rare to find up to 75% of the word occurrences in its negated
sense – which may introduce an unacceptable amount of sense misidentification to the textanalytical endeavor.
In fact, analysis of the current corpus has found that the negation of words like accept, agree,
trust, comfortable and ready is substantial enough to add to the reliability of the dictionary of
negative emotions, while the negation of words like bother contributes similarly to the reliability
of the positive emotion dictionary.
The results of this study may shed a new light on findings made with traditional textanalytical systems. For example, using the LIWC text-analytical program, low to no correlations
were found between measures of emotional dysregulation and the frequency of emotion words
used by the informants. Thus, Pennebaker & King (1999) found that in a sample of 841
undergraduate students, self-report measures of Neuroticism correlated 0.16 with negative
emotion words and -0.13 with positive emotion words in their personal essays. Both correlations
were statistically significant and both exhibited the expected direction, but their effect size was
not consistent with the overwhelming array of models that identify negativity bias as the core
bias underlying Neuroticism or emotional dysregulation. Similar meager magnitude of
correlation was found between negative emotion words in narratives concerning the September
11 terrorism attack and self-reported neuroticism of the speakers, while no statistically
Measurement of Negativity Bias 19
significant correlation was established between speakers’ Neuroticism and positive emotion
word frequency (Williams et al., 2003). Consistent with these results, in comparing poetry
samples from poets who committed suicide to those who did not, Stirman & Pennebaker (2001)
found no statistically significant effect of negative or positive emotion word frequency in the
poetry excerpts and the suicidality of the poet.
Somewhat surprisingly, the above-mentioned studies, among numerous others, found
associations between measures of emotional distress and personal pronouns that were equal or
even higher in their magnitude than those between emotional distress and negativity bias in the
use of emotion words. For example, Pennebaker & King (1999) found a correlation of 0.13
between Neuroticism and first person pronoun, while Stirman & Pennebaker (2001), using
ANOVA methodology, found that elevated first person markers and lower first person plural
markers are associated with risk for suicide in poetry excerpts. In a similarly naturalistic speech
samples, Pennebaker and Lay (2002) found that New York City’s Mayor Rudy Guilani’s use of
first-person singular, and not of negative emotion words, was consistently greater during times of
personal distress. Several other studies using the LIWC found that pronouns were also
meaningfully related to recovery from traumatic events, while emotion words were either
unpredictive of grief response following trauma or exhibiting surprising patterns. Thus, Boals &
Klein (2005) found that pronoun use in narratives of romantic breakup was highly associated
with a self-report measure of grief, unlike the use of negative or positive emotions, and in a textanalytical study of the evolvement of chat-rooms conversations concerning princess Diana
shortly after her death, negative emotions had not changed, while positive emotions were used in
significantly higher level than control chats. In contrast, first person pronouns have changed
steadily and predictably, with grieving writers using collective speech (i.e. high level of plural
Measurement of Negativity Bias 20
pronouns and low levels of singular pronouns) to individualized speech (Stone & Pennebaker,
2002).
The superior stability, consistency and relative robustness of the association between
indices of emotion dysregulation and pronouns as compared to their association with emotion
words led researchers to conclude that social-psychological and sociological theories of mental
health may be more pertinent than emotion regulation theories of mental health. For example,
Stirman & Pennebaker (2001) privileged Durkheim’s social integration theory of suicide over
hopelessness theories of suicide, citing the clear association between higher first person singular
markers and lower first person plural markers in poets who committed suicide, compared to no
main effect of negative or positive emotion on suicidality. Similarly, theories of grief highlighted
the collectivistic vs. individualized aspects of the phenomenon as well as the cognitive search for
meaning, and downplayed the role of emotional experience as expressed by emotional language.
With LIWC being the sole software of choice for text-analysis from 1999 onwards,
numerous publications have since ‘confirmed’ the privileged role of personal pronouns in
emotion dysregulation, above and beyond the expression of emotions. This impressive body of
consistent findings created a sense of consensus regarding the relatively marginal role of
expressive emotionality in emotion dysregulation, thus contributing to the phenomenological rift
between clinical theory and practice, which focus on emotional experience, and social
psychological theories of emotion dysregulation that favor constructs as self-involvement, social
embeddedness, and repressive coping style as the most defensible empirically validated
processes underlying emotion dysregulation (other LIWC-based studies that reported the same
pattern include: Campbell & Pennebaker, 2003; Gill, 2003; Oberlander & Gill, 2004; Bohanek et
al., 2005; Simmons et al., 2005; Mairesse et al., 2007, among others).
Measurement of Negativity Bias 21
The robustly augmented associations between the corpus-based text-analytical
dictionaries reported here and measures of mental distress urge caution in accepting uncritically
the above mentioned body of research, however extensive and diverse. As Table 1 shows,
introducing context to the internal dictionary changes dramatically its ability to detect
emotionality, leading to significantly higher correlations with mental health indices (see Table 2
for statistical significance of difference). Further, since emotions are usually attributed to
persons, with several emotions being attributable or relevant to one person, it comes as no
surprise that pronouns are correlated with emotional processes or states. Further, the grammatical
rules concerning pronouns ensure their relative ‘inertness’ to context compared to emotions.
While we have a host of linguistic qualifiers to emotions (e.g. “very happy”, “kind of happy”,
“not at all happy”, etc.), no similar contextual operators exist that could change the basic
meaning of pronouns. Therefore, basic grammatical rules may explain both the involvement of
pronouns in emotional narratives as well as its consistency compared to those of emotion words.
However, using corpus-based emotional dictionary may reveal different relative significance of
emotional language compared to pronouns, when narratives about emotional processes or states
are concerned.
The cumulative experience with other text-analytical studies adds to the caution with which the
results of this study should be interpreted. As corpus-based scales, the proposed dictionaries may
be of limited generalizability due to a plethora of factors, including: corpus-size, verbal
exclusivity, speech genre and speaker demographics.
Despite the fact that the corpus in this study included close to 500 participants and
380,000 words, past socio-linguistics research has demonstrated inconsistencies in the estimation
of word frequencies that persisted up to corpus-size of xxx words. To assess the stability of the
Measurement of Negativity Bias 22
corpus, the preliminary construction of the dictionaries was based on keyword concordances of
only half of the narratives, while its validation was conducted using the other half of the corpus.
Added stability to corpus-based measures comes from the use of dictionaries, or aggregates of
words, to estimate emotionality.
Text-analytical scales that are based on orthographic transcription of speech can only
detect emotionality if it is explicitly verbalized. Thus, emotional information conveyed by nonverbal means such as intonation and prosody, however significant, is left undetected. Similarly
undetectable is the avoidance of explicitly verbalized emotionality (e.g. irony, sarcasm,
idiosyncratic figurative language etc.), which can also serve as an effective tool for expressing
emotions (e.g. Labov, 1982). In the dictionaries proposed here, laugher and cry were the only
non-verbal emotionality markers that were audible and unambiguous, and their inclusion indeed
contributed to the accuracy of the dictionaries. Since non-verbal emotional cues are used more in
familiar, informal and interactive settings (e.g. Brown & Levinson, 1987; Biber, 1995; Clark,
1996; Hudson, 1996; Baugh, 2001; Coupland, 2001; Eckert & Rickford, 2001), the current
elicitation conditions may have minimized their effect and thus contributed to the dictionaries
sensitivity. The proposed dictionaries, conversely, may prove less sensitive in different
interaction settings.
Lastly, following speech variations, the sensitivity of corpus-based measures may vary
with the demographic characteristics of the speakers, including geographic background (e.g.
Coupland, 2001; Ash, 2003), race (e.g. Mufwene et al., 1998), Socio-economic status, (e.g.
Labov, 1972; Wolfram & Fasold, 1974), age and generation (e.g. Snow & Hoefnagel-Hoehle,
1978; Bloom, 1994; Eckert & Rickford, 2001; Tagliamonte & D'Arcy, 2007), and gender
(Coates, 1998; Cohen, in press).
Measurement of Negativity Bias 23
Although concerns regarding the exhaustiveness and generalizability of corpus-based
linguistic measures are the first set of unknowns to study, the current dictionaries have already
demonstrated the importance of context, however narrow, in determining the sense in which an
emotion word is being used. The robust, highly statistically significant and valence-consistent
correlations between the corpus-based emotion dictionaries and measures of psychological
distress and dysphoria has the potential of rekindling the search for theory-driven assessment of
emotion dysregulation. Better estimation of emotional experience also holds a promise for a
more accurate evaluation of the relative contribution that self-involvement, social embeddedness,
referential activity and other constructs that rely on the context-independent nature of speech
particles (e.g. Spencer & Spencer, 1993; Bucci, 1997, 2001; Stirman & Pennebaker, 2001;
Campbell & Pennebaker, 2003; Pennebaker et al., 2003; Fertuck et al., 2004) play in emotion
dysregulation.
Measurement of Negativity Bias 24
References
Abramson, L. Y., Metalsky, G. I., & Alloy, L. B. (1989). Hopelessness depression: A theorybased subtype of depression. Psychological review, 96(2), 358-372.
Alloy, L. B., Abramson, L. Y., Whitehouse, W. G., Hogan, M. E., Tashman, N. A., & Steinberg,
D. L. et al. (1999). Depressogenic cognitive styles: Predictive validity, information
processing and personality characteristics, and developmental origins. Behaviour
Research & Therapy, 37(6), 503-531.
Alpers, G. W.,Winzelberg, A. J., Classen, C., Roberts, H., Dev, P., Koopman, C., and Taylor, C.
B. (2005). Evaluation of computerized text analysis in an Internet breast cancer support
group. Computers in Human Behavior, 21, 343–358
Anderson, N. H. (1968). Likableness ratings of 555 personality-trait words. Journal of
Personality & Social Psychology, 9(3), 272-279.
Ash, S. (2003). A national survey of North American dialects. American speech, suppl. 88, 5773.
Baldwin, M. W. (1992). Relationship schemas and the processing of social information.
Psychological Bulletin. 112, 461-484.
Bargh, J. A., & Tota, M. E. (1988). Context-dependent automatic processing in depression:
Accessibility of negative constructs with regard to self but not others. Journal of
Personality & Social Psychology, 54(6), 925-939.
Baugh, J. (2001). A dissection of style-shifting. In P. Eckert & J. R. Rickford (Eds.), Style and
sociolinguistic variation. (pp. 109-118). Cambridge, MA: Cambridge University Press.
Measurement of Negativity Bias 25
Beck, A.T. (1976). Cognitive therapies and the emotional disorders. New York: International
University Press.
Beck, A. T. (2002). Cognitive models of depression. In R. L. Leahy, & E. T. Dowd (Eds.),
Clinical advances in cognitive psychotherapy: Theory and application (pp. 29-61). New
York, NY, US: Springer Publishing Co.
Beck, A. T., Epstein, N., & Harrison, R. (1983). Cognitions, attitudes and personality dimensions
in depression. British Journal of Cognitive Psychotherapy, 1(1), 1-16.
Beck, A. T., Steer, R. A., & Garbin, M. G. (1988). Psychometric properties of the beck
depression inventory: Twenty-five years of evaluation. Clinical psychology review, 8(1),
77-100.
Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for
measuring depression. Archives of General Psychiatry, 4, 561-571.
Bell, A. (1984). Language style as audience design. Language in society, 13(2), 145-204.
Biber, D. (1995). Dimensions of register variation: A cross-linguistic comparison. Cambridge:
Cambridge University Press.
Biber, D., Conrad, S., & Reppen, R. (1998). Corpus Linguistics: Investigating language
structure and use. Cambridge, MA: Cambridge University Press.
Blaney, P. H. (1986). Affect and memory: A review. Psychological bulletin, 99, 2, 229-246.
Bloom, P. (1994). Overview: Controversies in language acquisition. In P. Bloom (Ed.),
Language acquisition: Core readings (pp. 5-15). Cambridge, MA: MIT Press.
Boals, A. & Klein, K. (2005). Word Use in Emotional Narratives about Failed Romantic
Relationships and Subsequent Mental Health. Journal of Language and Social
Psychology, 24, 252-268
Measurement of Negativity Bias 26
Bohanek, J. G., Fivush, R., & Walker, E. (2005). Memories of positive and negative emotional
events. Applied Cognitive Psychology, 19(1), 51-66.
Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage.
Cambridge, MA: Cambridge University Press.
Bucci, W. (1997). Psychoanalysis and cognitive science: A multiple code theory. New York,
NY, US: Guilford Press.
Bucci, W. (2001). Pathways of emotional communication. Psychoanalytic Inquiry, 21, 1, 40-70.
Bucci, W., & Freedman, N. (1981). The language of depression. Bulletin of the Menninger
Clinic, 45, 4, 334-358.
Cacioppo, J. T., von Hippel, W., & Ernst, J. M. (1997). Mapping cognitive structures and
processes through verbal content: The thought-listing technique. Journal of Consulting &
Clinical Psychology, 65, 6, 928-940.
Campbell, R. S., & Pennebaker, J. W. (2003). The secret life of pronouns: Flexibility in writing
style and physical health. Psychological Science, 14, 1, 60-65.
Chien, A. J., & Dunner, D. L. (1996). The tridimensional personality questionnaire in
depression: State versus trait issues. Journal of Psychiatric Research, 30, 21-27.
Clark, D. A., Beck, A. T., & Alford, B. A. (1999). Scientific foundations of cognitive theory and
therapy of depression. New York, NY, US: John Wiley & Sons, Inc.
Clark, H. H. (1996). Using language. Cambridge, MA: Cambridge University Press.
Clark, D. A. (1988). The validity of measures of cognition: A review of the literature. Cognitive
Therapy and Research, 12, 1-20.
Coates, J. (ed.) (1998) Language and Gender: A Reader. Blackwell: Oxford, Malden Masc.
Measurement of Negativity Bias 27
Cohen, S. J. (In press). Gender differences in speech temporal patterns detected using lagged cooccurrence text-analysis of personal narratives. Journal of Psycholinguistics Research
Collins, N. L., & Feeney, B. C. (2004). Working models of attachment shape perceptions of
social support: Evidence from experimental and observational studies. Journal of
Personality & Social Psychology, 87, 3, 363-383.
Coupland, N. (2001). Language, situation, and the relational self: Theorizing dialect-style in
sociolinguistics. In P. Eckert & J. R. Rickford (Eds.), Style and sociolinguistic variation
(pp. 185-210). Cambridge, MA: Cambridge University Press.
Cyr, J. J., McKenna-Foley, J. M., & Peacock, E. (1985). Factor structure of the SCL-90-R: Is
there one? Journal of personality assessment, 49, 6, 571-578.
Dahl, H., & Stengel, B. (1978). A classification of emotion words: A modification and partial
test of de rivera's decision theory of emotions. Psychoanalysis & Contemporary Thought,
1, 2, 269-312.
Davison, G. C., Vogel, R. S., & Coffman, S. G. (1997). Think-aloud approaches to cognitive
assessment and the articulated thoughts in simulated situations paradigm. Journal of
Consulting & Clinical Psychology, 65, 6, 950-958.
Demorest, A., Crits-Christoph, P., Hatch, M., & Luborsky, L. (1999). A comparison of
interpersonal scripts in clinically depressed versus nondepressed individuals. Journal of
Research in Personality, 33, 3, 265-280.
Derogatis, L. R., Rickels, K., & Rock, A. F. (1976). The SCL-90 and the MMPI: A step in the
validation of a new self-report scale. British Journal of Psychiatry, 128, 280-289.
Measurement of Negativity Bias 28
Derogatis, L. R., & Savitz, K. L. (2000). The SCL-90-R and Brief Symptom Inventory (BSI) in
primary care. In M. E. Maruish (Ed.), Handbook of psychological assessment in primary
care settings (pp. 297-334). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Dobson, K. S., & Breiter, H. J. (1983). Cognitive assessment of depression: Reliability and
validity of three measures. Journal of Abnormal Psychology, 92, 107–109.
Eckert, P., & Rickford, J. R. (Eds.). (2001). Style and sociolinguistic variation. Cambridge, MA:
Cambridge University Press.
Endler, N. S., Denisoff, E., & Rutherford, A. (1998). Anxiety and depression: Evidence for the
differentiation of commonly co-occurring constructs. Journal of Psychopathology &
Behavioral Assessment, 20, 2, 149-171.
Fertuck, E. A., Bucci, W., Blatt, S. J., & Ford, R. Q. (2004). Verbal representation and
therapeutic change in anaclitic and introjective inpatients. Psychotherapy: Theory,
Research, Practice, Training, 41, 1, 13-25.
Fresco, D. M., Heimberg, R. G., Abramowitz, A., & Bertram, T. L. (2006). The effect of a
negative mood priming challenge on dysfunctional attitudes, explanatory style, and
explanatory flexibility. British Journal of Clinical Psychology, 45,2, 167-183.
Giles, H. (2001). Sociopsychological parameters of style shifting. In P. Eckert & J. R. Rickford
(Eds.), Style and sociolinguistic variation (pp. 211-219). Cambridge, MA: Cambridge
University Press.
Gill, A. (2003). Personality and Language: The Projection and Perception of Personality in
Computer-Mediated Communication. Ph.D. thesis, University of Edinburgh.
Glass, C. R., & Arnoff, D. B. (1997). Questionnaire methods of cognitive self-statement
assessment. Journal of Consulting and Clinical Psychology, 65, 911-927.
Measurement of Negativity Bias 29
Gotlib, I. H. (1984). Depression and general psychopathology in university students. Journal of
Abnormal Psychology, 93, 1, 19-30.
Gotlib, I. H., Lewinson, P. M., & Seeley, J. R. (1995). Symptoms versus a diagnosis of
depression: Differences in psychosocial functioning. Journal of Consulting and Clinical
Psychology, 63, 90-100
Gottschalk, L. (1997). The unobstrusive measure of psychological traits and states. In C. W.
Roberts (Ed.), Text analysis for the social sciences: Methods for drawing statistical
inferences from texts and transcripts (pp. 117-129). Mahwah, NJ: Lawrence Erlbaum.
Gottschalk, L. A., & Bechtel, R. J. (1982). The measurement of anxiety through the computer
analysis of verbal samples. Comprehensive Psychiatry, 23, 4, 364-369.
Gottschalk, L. A., & Gleser, G. C. (1969). The measurement of psychological states through the
content analysis of verbal behavior. Oxford, England: U. California Press.
Haaga, D. A., Dyck, M. J., & Ernst, D. (1991). Empirical status of cognitive theory of
depression. Psychological Bulletin, 110, 2, 215-236.
Haeffel, G. J., Abramson, L. Y., Voelz, Z. R., Metalsky, G. I., Halberstadt, L., Dykman, B. M., et
al. (2005). Negative cognitive styles, dysfunctional attitudes, and the remitted depression
paradigm: A search for the elusive cognitive vulnerability to depression factor among
remitted depressives. Emotion, 5, 343 – 348.
Hartman-Hall, H. M. & Haaga, D. A. F. (1999) Content Analysis of Cognitive Bias:
Development of a Standardized Measure. Journal of Rational Emotive and Cognitive
Behavior Therapy, 17, 2, 105-114
Hermans, H. J. M. (1995). From assessment to change: The personal meaning of clinical
problems in the context of the self-narrative. In R. A. Neimeyer, & M. J. Mahoney (Eds.),
Measurement of Negativity Bias 30
Constructivism in psychotherapy. (pp. 247-272). Washington, DC, US: American
Psychological Association;
Hill, A. B., Kemp-Wheeler, S. M., & Jones, S. A. (1986). What does the beck depression
inventory measure in students? Personality & Individual Differences, 7, 1, 39-47.
Horowitz, L. M. (1979). On the cognitive structure of interpersonal problems treated in
psychotherapy. Journal of Consulting & Clinical Psychology, 47, 1, 5-15.
Hudson, R. A. (1996). Sociolinguistics. New York, NY: Cambridge University Press.
Hurlburt, R. T. (1980). Validation and correlation of thought sampling with retrospective
measures. Cognitive Therapy & Research, 4, 2, 235-238.
Hurlburt, R. T. (1997). Randomly sampling thinking in the natural environment. Journal of
Consulting and Clinical Psychology, 65, 941–949.
Ingram, R. E. (1984). Toward an information-processing analysis of depression. Cognitive
Therapy & Research, 8, 5, 443-477.
Ingram, R. E , Kendall, P. C , Siegle, G., Guarino, J., & McLaughlin, S. (1995). Psychometric
properties of the Positive Automatic Thoughts Questionnaire. Psychological Assessment,
7, 495-507.
Isen, A. M. (1984). Toward understanding the role of affect in cognition. In R. S. Wyer & T. K.
Srull (Eds.), Handbook of social cognition. (pp. 179-236). Hillsdale, NJ: Lawrence
Erlbaum Associates.
Kelly, E. F., & Stone, P. J. (1975). Computer recognition of English word senses. Amsterdam,
Netherlands: North Holland Publishing Company.
Klein, M. H., Mathieu-Coughlan, P., & Kiesler, D. J. (1986). The experiencing scales. In L. S.
Greenberg, & W. M. Pinsof (Eds.), The psychotherapeutic process: A research
Measurement of Negativity Bias 31
handbook. Guilford clinical psychology and psychotherapy series (pp. 21-71). New York,
NY, US: Guilford Press.
Kuiper, N. A., & Higgins, E. T. (1985). Social cognition and depression: A general integrative
perspective. Social Cognition. Special issue: Depression, 3, 1, 1-15.
Kuiper, N. A., Olinger, L. J., & MacDonald, M. R. (1988). Vulnerability and episodic cognitions
in a self-worth contingency model of depression. In L. B. Alloy (Ed.), Cognitive
processes in depression. (pp. 289-309). New York, NY, US: Guilford Press.
Labov, W. (1972). Sociolinguistic patterns. Philadelphia: University of Pennsylvania Press.
Labov, W. (1982). Speech Actions and Reactions in Personal Narrative. In Analyzing Discourse:
Text and Talk, D. Tannen (ed.). Washington D.C.: Georgetown University Press.
Lee, F., & Peterson, C. (1997). Content analysis of archival data. Journal of Consulting &
Clinical Psychology, 65, 6, 959-969.
Lindeman, M., & Verkasalo, M. (1995). Personality, situation, and positive – negative
asymmetry in socially desirable responding. European Journal of Personality, 9, 125 –
134.
Luborsky, L., & Crits-Christoph, P. (1990). Understanding transference: The core conflictual
relationship theme method. New York, NY, US: Basic Books, Inc.
Mairesse, F., Walker, M., Mehl, M. & Moore, R. (2007). Using linguistic cues for the automatic
recognition of personality in conversation and text. Journal of Artificial Intelligence
Research, 30, 457–500.
Mangan, G. L., & Hookway, D. (1988). Perception and recall of aversive material as a function
of personality type. Personality & Individual Differences, 9, 2, 289-295.
Measurement of Negativity Bias 32
Marchitelli, L. (1983). The language of the emotions: Couples in conversation. Unpublished
M.A., University of California at Berkeley, Berkeley.
Marchitelli, L. & Levenson, R.W. (1992). When couples converse: the language and physiology
of emotion. Paper presented at the Society for Psychophysiological Research, San Diego,
CA.
Markus, H. (1977). Self-schemata and processing information about the self. Journal of
Personality & Social Psychology, 35, 2, 63-78.
Martin, M. (1985). Neuroticism as predisposition toward depression: A cognitive mechanism.
Personality & Individual Differences, 6, 3, 353-365.
Martin, M., Ward, J. C., & Clark, D. M. (1983). Neuroticism and the recall of positive and
negative personality information. Behaviour Research & Therapy, 21, 5, 495-503.
Mathews, A., & MacLeod, C. (1994). Cognitive approaches to emotion and emotional disorders.
Annual Review of Psychology, 45, 25-50.
Mergenthaler, E., & Bucci, W. (1999). Linking verbal and non-verbal representations: Computer
analysis of referential activity. British Journal of Medical Psychology, 72, 3, 339-354.
Mergenthaler, E., & Stinson, C. H. (1992). Psychotherapy transcription standards. Psychotherapy
Research, 2, 2, 125-142.
Mufwene, S.S., Rickford, J.R., Bailey, G., & Baugh, J. (Eds.). (1998). African-American
English: Structure, history and use. London: Routledge.
Nasby, W., & Kihlstrom, J. F. (1986). Cognitive assessment of personality and psychopathology.
In R. E. Ingram (Ed.), Information processing approaches to clinical psychology. (pp.
217-239). San Diego, CA, US: Academic Press, Inc.
Measurement of Negativity Bias 33
Nosek, B. A. (2005). Moderators of the relationship between implicit and explicit evaluation.
Journal of Experimental Psychology, 134, 565- 584.
Oberlander, J. and Gill, A. (2004). Language generation and personality: two dimensions, two
stages, two hemispheres? In Papers from the AAAI Spring Symposium on Architectures
for Modeling Emotion: Cross-Disciplinary Foundations, 104– 111.
Oliver, J. M., & Baumgart, E. P. (1985). The Dysfunctional Attitude Scale: psychometric
properties and relation to depression in an unselected adult population. Cognitive Therapy
and Research, 11, 25–40.
Olkin, I., & Finn, J. D. (1990). Testing correlated correlations. Psychological Bulletin, 108, 2,
330-333.
Olkin, I., & Finn, J. D. (1995). Correlations redux. Psychological Bulletin, 118, 1, 155-164.
Oxman, T. E., Rosenberg, S. D., Schnurr, P. P., & Tucker, G. J. (1988). The language of altered
states. Journal of Nervous & Mental Disease, 176, 7, 401-408.
Paulhus, D. L., & John, O. P. (1998). Egoistic and moralistic biases in self-perception: The
interplay of self-deceptive styles with basic traits and motives. Journal of Personality, 66,
1025 – 1060.
Pennebaker, J. W., & Francis, M. E. (1996). Cognitive, emotional, and language processes in
disclosure. Cognition & Emotion, 10, 6, 601-626.
Pennebaker, J. W., & Francis, M. E. (1999). Linguistic inquiry and word count: LIWC [Software
Manual]. Mahwah, NJ: Erlbaum publishers.
Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word count:
LIWC 2001 [Software Manual]. Mahwah, NJ: Erlbaum publishers.
Measurement of Negativity Bias 34
Pennebaker, J. W., & Graybeal, A. (2001). Patterns of natural language use: Disclosure,
personality, and social integration. Current Directions in Psychological Science, 10, 3,
90-93.
Pennebaker, J. W., & King, L. A. (1999). Linguistic styles: Language use as an individual
difference. Journal of Personality & Social Psychology, 77, 6, 1296-1312.
Pennebaker, J. W., & Lay, T. C. (2002). Language use and personality during crises: Analyses of
mayor rudolph giuliani's press conferences. Journal of Research in Personality, 36, 3,
271-282.
Pennebaker, J. W., & Stone, L. D. (2003). Words of wisdom: Language use over the life span.
Journal of Personality & Social Psychology, 85, 2, 291-301.
Pennebaker, J. W., Mayne, T. J., & Francis, M. E. (1997). Linguistic predictors of adaptive
bereavement. Journal of Personality & Social Psychology, 72, 4, 863-871.
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psychological aspects of natural
language use: Our words, our selves. Annual Review of Psychology, 54, 547-577.
Pennebaker, J. W., Zech, E., & Rimé, B. (2001). Disclosing and sharing emotion: Psychological,
social, and health consequences. In M. S. Stroebe, & R. O. Hansson (Eds.), Handbook of
bereavement research: Consequences, coping, and care. (pp. 517-543). Washington, DC,
US: American Psychological Association
Persons, J. B., & Miranda, J. (1992). Cognitive theories of vulnerability to depression:
Reconciling negative evidence. Cognitive Therapy and Research, 16, 485–502.
Peterson, C. (1992). Explanatory style. In C. P. Smith (Eds.), Handbook of thematic content
analysis (pp. 376-382). New York: Cambridge.
Polaino, A., & Senra, C. (1991). Measurement of depression: Comparison between
Measurement of Negativity Bias 35
self-reports and clinical assessments of depressed outpatients. Journal of Psychopathology and
Behavioral Assessment, 13, 313-324.
Rector, N. A., Segal, Z. V., & Gemar, M. (1998). Schema research in depression: A canadian
perspective. Canadian Journal of Behavioural Science.Special Canadian perspectives on
research in depression, 30, 4, 213-224.
Ritterband, L. M., & Spielberger, C. D. (1996). Construct validity of the Beck Depression
Inventory as a measure of state and trait depression in nonclinical populations.
Depression and Stress, 2, 123-145.
Rosenberg, S. D., Blatt, S. J., Oxman, T. E., & McHugo, G. J. (1994). Assessment of object
relatedness through a lexical content analysis of the TAT. Journal of personality
assessment, 63, 2, 345-362.
Rosenberg, S. D., Schnurr, P. P., & Oxman, T. E. (1990). Content analysis: A comparison of
manual and computerized systems. Journal of personality assessment, 54, 1, 298-310.
Rude, S., Covich, J., Jarrold, W., Hedlund, S., & Zentner, M. (2001). Detecting depressive
schemata in vulnerable individuals: questionnaires versus laboratory tasks. Cognitive
Therapy and Research, 25, 103–116.
Rude, S. S., Valdez, C. R., Odom, S., & Ebrahimi, A. (2003). Negative cognitive biases predict
subsequent depression. Cognitive Therapy & Research, 27, 4, 415-429.
Ruiz-Caballero, J. A., & Bermúdez, J. (1995). Neuroticism, mood, and retrieval of negative
personal memories. Journal of General Psychology, 122, 1, 29-35.
Rusting, C. L. (1998). Personality, mood, and cognitive processing of emotional information:
Three conceptual frameworks. Psychological bulletin, 124, 2, 165-196.
Measurement of Negativity Bias 36
Rusting, C. L. (1999). Interactive effects of personality and mood on emotion-congruent memory
and judgment. Journal of Personality & Social Psychology, 77, 5, 1073-1086.
Scher, C. D., Segal, Z. V., & Ingram, R. E. (2004). Beck's theory of depression: Origins,
empirical status, and future directions for cognitive vulnerability. In R. L. Leahy (Ed.),
Contemporary cognitive therapy: Theory, research, and practice. (pp. 27-44). New York,
NY, US: Guilford Press.
Segal, H. G., Westen, D., Lohr, N. E., & Silk, K. R. (1993). Clinical assessment of object
relations and social cognition using stories told to the picture arrangement subtest of the
WAIS--R. Journal of personality assessment, 61, 1, 58-80.
Segal, Z. V., & Ingram, R. E. (1994). Mood priming and construct activation in tests of cognitive
vulnerability to unipolar depression. Clinical psychology review, 14, 7, 663-695.
Shaw, B. F., & Dobson, K. S. (1981). The cognitive assessment of depression. In T. Merluzzi, C.
Glass, & M. Genest (Eds.), Cognitive assessment. New York: Guilford Press.
Shedler, J., Mayman, M., & Manis, M. (1993). The illusion of mental health. American
Psychologist, 48, 1117 – 1131.
Shepard, C. A., Giles, H., & Le Poire, B. A. (2001). Communication accomodation theory. In P.
W. Robinson & H. Giles (Eds.), The new handbook of language and social psychology
(pp. 33-56). New York, NY: John Wiley & Sons.
Simmons, R. A., & Gordon, P. C., Chambless, D. L. (2005). Pronouns in marital interaction:
What do ‘you’ and ‘I’ say about marital health. Psychological Journal, 16, 932-936.
Sinclair, J. (1991). Corpus, Concordance, Collocation. Oxford, UK: Oxford University Press.
Snow, C. E., & Hoefnagel-Hohle, M. (1978). The critical period for language acquisition:
Evidence from second language learning. Child Development, 49, 4, 1114-1128.
Measurement of Negativity Bias 37
Spencer, L. M., & Spencer, S. M. (1993). Competence at work: Models for superior
performance. New York:Wiley
Stirman, S. W., & Pennebaker, J. W. (2001). Word use in the poetry of suicidal and nonsuicidal
poets. Psychosomatic medicine, 63, 4, 517-522.
Stone, A. A., Turkkan, J. S., Bachrach, C. A., Jobe, J. B., Kurtzman, H. S., & Cain, V. S. (Eds.).
(2000). The science of self-report: Implications for research and practice. Mahwah, NJ:
Lawrence Erlbaum Associates.
Stone, L. D., & Pennebaker, J. W. (2002). Trauma in real time: Talking and avoiding online
conversations about the death of Princess Diana. Basic and Applied Social Psychology,
24, 3, 173-183
Stone, P. J. (1997). Thematic text analysis: New agendas for analyzing text content. In C. W.
Roberts (Ed.), Text analysis for the social sciences: Methods for drawing statistical
inferences from texts and transcripts. (pp. 35-53). Mahwah, NJ: Lawrence Erlbaum
Associates.
Stone, P. J., Dunphy, D. C., & Smith, M. S. (1966). The general inquirer: A computer approach
to content analysis. Oxford, England: M.I.T. Press.
Sweeney, K., & Whissell, C. (1984). A dictionary of affect in language: I. establishment and
preliminary validation. Perceptual & Motor Skills, 59, 3, 695-698.
Tagliamonte, S. A., & D'Arcy, A. (2007). Frequency and variation in the community grammar:
Tracking a new change through the generations. Language Variation and Change, 19, 2,
199-217.
Teasdale, J. D. (1999). Emotional processing, three modes of mind and the prevention of relapse
in depression. Behaviour Research & Therapy. Special Issue: Cognitive Behaviour
Measurement of Negativity Bias 38
Therapy: Evolution and prospects.A festschrift in honour of Dr S.Rachman, Editor of
Behavior Research and Therapy, 37(Suppl 1), S53-S77.
Viney, L. L. (1983). The assessment of psychological states through content analysis of verbal
communications. Psychological bulletin, 94, 3, 542-563.
Watkins, J., & Rush, A. J. (1983). The Cognitive Response Test. Cognitive Therapy and
Research, 7, 425-436.
Watson, D., & Clark, L. A. (1984). Negative affectivity: The disposition to experience aversive
emotional states. Psychological bulletin, 96, 3, 465-490.
Weintraub, W. (1989). Verbal behavior in everyday life. New York, NY, US: Springer
Publishing Co;
Wenzlaff, E. M., & Wegner, D. M. (2000). Thought suppression. Annual Review of Psychology,
51, 59–91.
Whissell, C. M., Fournier, M., Pelland, R., & Weir, D. (1986). A dictionary of affect in
language: IV. reliability, validity, and applications. Perceptual & Motor Skills, 62, 3,
875-888.
Williams, K. M., Paulhus, D. L., & Nathanson, C. (2003, May). Personality Correlates of
Emotional Reactions to 9/11. Poster presented at the 83rd Annual Convention of the
Western Psychological Association, Vancouver, BC, Canada.
Winters, N., Myers, K., & Proud, L. (2002). Ten-year review of rating scales, III: Assessing
suicidality, cognitive style, and self-esteem. Journal of American Academy of Child &
Adolescent Psychiatry, 41, 1150 – 1181.
Winters, K. C., & Neale, J. M. (1985). Mania and low self-esteem. Journal of Abnormal
Psychology, 94, 282-290
Measurement of Negativity Bias 39
Wolfram, W., & Fasold, R. W. (1974). The study of social dialects in American English.
Englewood Cliffs, NJ: Prentice-Hall.
Zuckerman, M., Lubin, B., & Robins, S. (1965). Validation of the multiple affect adjective check
list in clinical situations. Journal of consulting psychology, 29, 6, 594.
Table 1
Pearson correlations between the 3 computerized text-analysis systems and various measures of mental health
Dictionary
1
2
3
4
5
6
7
N=483
1. General Severity Index (GSI)
--
2. Beck Depression Inventory
(BDI)
.76**
--
3. Corpus-based negative emotion
.41**
.41**
4. Corpus-based positive emotion
-.27**
-.26**
5. General Inquirer negative
emotion
.08
.05
6. General Inquirer positive
emotion
-.04
-.01
7. LIWC negative emotion
.04
.11*
--.17**
--
.28**
-.06
.05
.46**
.24**
--
-.04
-.05
-.00
--
.50**
8. LIWC positive emotion
.02
.01
.06
.05
-.03
.01
.30**
Note: *) p<0.05; **) p<0.01
40
Table 2
95% Confidence intervals for the differences between the correlations of the
text-analytical scales and the outcome measures, as calculated by Olkin &
Finn (1990,1995) procedure.
Correlation difference
Correlations
Correlations
with GSI
with BDI
N=483
Corpus-based negative emotion -
0.331±0.005
0.362±0.005
0.369±0.004
0.303±0.004
-0.228±0.006
-0.251±0.006
-0.293±0.007
-0.272±0.007
General Inquirer negative emotion
Corpus-based negative emotion – LIWC
negative emotion
Corpus-based positive emotion General Inquirer positive emotion
Corpus-based positive emotion – LIWC
positive emotion