DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Jeffrey J. Rachlinski,* Sheri Lynn Johnson,** Andrew J. Wistrich*** & Chris Guthrie****
Abstract
Unconscious racism appears to be widespread.
Even people who embrace
egalitarian norms harbor invidious implicit associations about African Americans. But
do these unconscious biases affect the judgment of significant decision makers in our
society?
To begin to answer this question, we measured the influence of implicit
associations on legal judgments made by 133 sitting trial judges. We had judges assess
three hypothetical scenarios in which the race of the defendant varied and correlated
these assessments with judges’ implicit associations concerning race. We manipulated
the race of the defendant in two different ways: first by subliminally priming judges with
words associated with African-American and second by explicitly identifying the
defendant’s race. We found that, like other Americans, the judges held invidious implicit
associations concerning African Americans.
These associations, however, did not
influence their judgment when we explicitly identified the race of the defendant. In
contrast, those implicit associations did influence their judgment when we manipulated
the race of the defendant through subliminal techniques. These results suggest that
judges can control the influence of unconscious racial bias—but only when they are
consciously focused on doing so.
*
Professor of Law, Cornell Law School.
Professor of Law, Cornell Law School.
***
United States Magistrate Judge, Central District of California.
****
Associate Dean and Professor of Law, Vanderbilt Law School. The authors are grateful for the
comments and assistance of Ian Ayres, Jack Glaser, Tony Greenwald, Matthew Patrick Henry, Reid Hastie,
Christine Jolls, Dan Kahan, Jerry Kang, Cass Sunstein, and the participants in workshops at the University
of Arizona Law School; Brooklyn Law School; the University of Chicago Law School; Chicago-Kent Law
School; George Washington University Law School; Harvard Law School; the University of Illinois School
of Law; Notre Dame Law School; Ohio State University Law School,; Syracuse University Law School;
Villanova Law School; the University of Zurich; the Annual Meeting of the American Law and Economics
Association; the Annual Conference on Empirical Legal Studies.
**
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
I. Introduction
Race still matters in America.
African Americans are under-represented in
commencement ceremonies on college campuses, and in the executive suite, but overrepresented in the prison system, the welfare office, and the unemployment line.1
Nowhere are these racial disparities more troubling than in the criminal justice
system, where researchers have found evidence suggesting that African-American
defendants fare worse than their white counterparts. Consider just two examples: In a
recent study of bail-setting, researchers found that judges made more stringent judgments
concerning bail for African-American defendants than for similarly situated white
defendants.2 And in recent study of criminal sentencing, researchers found that judges
were more likely to deviate from the federal sentencing guidelines to impose harsher
sentences or “upward departures” on African-American defendants than on comparable
white defendants.3
What makes these studies particularly disturbing is that the
researchers relied upon methods that control for systematic differences between African
American and white defendants. Systematic structural gaps in factors such as education
and wealth of those who appear before the courts explain some of these disparities, but
these studies reveal that even holding these factors aside, the courts treat deal more
harshly with African American defendants than their white counterparts.
Understanding why racial disparities persist in the criminal justice system is vital.
Only if we understand why African-American defendants fare less well than similarly
situated white defendants can we determine how to remedy this deeply troubling
problem. Two potential sources of disparate treatment in court are explicit bias and
implicit bias.4 By explicit biases, we mean the kinds of bias that people knowingly—
sometimes openly—embrace. In contrast, we mean implicit bias to reflect stereotypical
associations so subtle that people who hold them might not even lack awareness of them.
1
See IAN AYRES, PERVASIVE PREJUDICE: UNCONVENTIONAL EVIDENCE OF RACE AND GENDER
DISCRIMINATION 1-4 (2002) (reviewing the evidence of continued racial disparities).
2
Ian Ayres & Joel Waldfogel, A Market Test for Race Discrimination in Bail Setting, 46 STAN. L. REV.
987 (1994).
3
David B. Mustard, Racial, Ethnic, and Gender Disparities in Sentencing: Evidence from the United
States Federal Courts, 44 J. L. & ECON. 285 (2001).
4
See Christine Jolls & Cass R. Sunstein, The Law of Implicit Bias, 94 CAL. L. REV. 969,969-70 (2006)
(providing examples of both explicit and implicit bias.)
2
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Surveys of American attitudes shows marked declines in explicit bias,5 but it still
might account for some of the racial disparities in the criminal justice system. The
expressed egalitarianism in surveys might be a fiction. People might simply misrepresent
their actual views in the surveys so as to conform to some perceived norm of
egalitarianism that does not actually exist. Furthermore, survey evidence shows that
some explicit bias persists.6 Moreover, even people who embrace egalitarian values
nevertheless hold attitudes that they believe support disparate treatment. 7 For example,
someone who harbors negative beliefs about the abilities of African-Americans is likely
to disfavor African-American job applicants, and to do so while proudly claiming to
make hiring decisions on the basis of merit.8 Explicit racism may have become less
overtly hostile than in decades past, but it might still affect how society functions.
But explicit bias is unlikely to account for the sizeable racial disparities in the
criminal justice system. Egalitarian norms are at their highest in the courts, where
professional norms and even professional rules demand equality.9 The disparities seen in
the studies of bail setting and sentencing reveal wide disparities between AfricanAmerican and white defendants that are unlikely to be the product of a small number of
bigoted judges.
Social psychologists have begun to identify empirical evidence that indicts
implicit biases as a likely cause of continued disparities.10 Their work shows that most
people who explicitly embrace egalitarian norms nevertheless harbor unconscious racial
associations about African Americans that might lead them to treat them in
5
See PAUL M. SNIDERMAN & THOMAS PIAZZA, BLACK PRIDE AND BLACK PREJUDICE 6-8 (2002).
See Id. at 8.
7
See JODY D. ARMOUR, NEGROPHOBIA AND REASONABLE RACISM: THE HIDDEN COSTS OF BEING BLACK
IN AMERICA 3-5 (1997).
8
See Id.; Thomas F. Pettigrew, The Ultimate Attribution Error: Extending Allport's Cognitive Analysis of
Prejudice, 5 PERSONALITY & SOC. PSYCHOL. BULL. 461 (1979).
9
See AMERICAN BAR ASSOCIATION, MODEL CODE OF JUDICIAL CONDUCT, Canon 7(2007) (“A judge shall
perform the duties of judicial office impartially and diligently.”)
10
Anthony G. Greenwald & Linda Hamilton Kreiger, Implicit Bias: Scientific Foundations, 94 CAL. L.
REV. 945, 950, 961 (2006) (“evidence that implicit attitudes produce discriminatory behavior is already
substantial and will continue to accumulate.”); Kirstin A. Lane, Jerry Kang & Mahzarin R. Banaji, Implicit
Social Cognition and Law, 3 ANN REV. L. & SOC. SCI. 427, 433 (2007) (“implicit social cognitions are
robust and pervasive”).
6
3
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
discriminatory ways.11
Subtle associations that most people harbor might facilitate
negative attributions that lead decisions makers to unwittingly place a thumb on the scale
against African Americans.12 If these implicit biases are as common among judges as
they are among the rest of us, they might account for the lion’s share of disparate
outcomes in the criminal justice system.
In this article, we report the results of the first-ever study of implicit bias in trial
judges. We seek both to determine whether judges have implicit biases of the type
observed among the general population and, importantly, whether those biases correlate
with the judges’ legal decision making.
We begin, in Part I, by examining the research on implicit bias and its impact on
behavior. In Part II we describe the methods of our study and in Part III we report our
results. In part IV we interpret the results and exploring the implications for the criminal
justice system. In Part V, we offer a brief conclusion. In brief, we find that judges
harbor the same pattern of invidious racial biases towards that most adults also harbor.
These biases, however, did not affect their judgment when they were aware of the race of
a criminal defendant. When we manipulated the race of defendants through subliminal
techniques, however, judges’ decisions correlated with their unconscious biases.
I.
Implicit Bias
Recent research in social cognition suggests that implicit biases might be
responsible for continuing racial disparities in society.13 To infer that implicit biases
account for racial disparities, researchers must first determine whether people have
implicit biases and then establish that implicit biases correspond to behaviors of interest.
A. Demonstrating Implicit Bias
See Jerry Kang & Mahzarin R. Banaji, Fair Measures: A Behavioral Realist Revision of “Affirmative
Action,” 94 CAL. L. REV. 1063, 1095, 1065 (2006) (arguing that implicit bias shows that “affirmative
action” programs are necessary to address “discrimination in the here and now”).
12
See id. at 1095-1100.
13
Jerry Kang, Trojan Horses of Race, 118 HARV. L. REV. 1489 (2005).
11
4
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
In their efforts to assess whether people possess implicit biases, social
psychologists have used a variety of methods, including subliminal priming techniques;14
studies of reaction times15; and even novel brain-imaging research.16 Standing front and
center among these methods, however, is the implicit association test or “IAT”.
Developed by a group of researchers led largely by Tony Greenwald, Mahzarin
Banaji, Brian Nosek, and others, the IAT applies decades of research from cognitive
psychology to the study of bias and stereotypes.17 The IAT takes different forms, but a
standard approach is to present participants with what appears to be a computer-based
sorting game that requires them to perform multiple trials of two different tasks:18
First, the researchers present participants with a computer screen that has the
words “white or good” in the upper left-hand corner of the screen and “black or bad” in
the upper right. The researchers then inform the participants that one of four types of
stimuli will appear in the center of the screen: white people’s faces, black people’s faces,
good (positive) words, and bad (negative) words. The researchers then explain that the
participants should press a designated key on the left side of the computer when a white
face or a good word appears and press a designated key on the right side of the computer
when a black face or a bad word appears. The participants complete several trials of this
first task.
Then, the computer is programmed to switch the spatial location of ‘good” and
“bad” so that the words “white or bad” appear in the upper left-hand corner and “black or
good” appear in the upper right. The researchers explain to the participants that they are
now supposed to press a designated key on the left side of the keyboard when a white
face or a bad word appears and press a designated key on the right side of the keyboard
14
John A. Bargh, The Four Horsemen of Automaticity: Awareness, Intention, Efficiency, and Control in
Social Cognition, in HANDBOOK OF SOCIAL COGNITION 1 (R. Wyer, & T. Srull eds., Lawrence Erlbaum
Associates, Inc. 1994).
15
See Greenwald & Kreiger, supra note 10, at 950-53 (labeling studies of implicit bias as studies of biases
in reaction times).
16
Elizabeth A. Phelps et al., Performance on Indirect Measures of Race Evaluation Predicts Amygdala
Activation, 12 J. COGNITIVE NEUROSCIENCE 1 (2000).
17
Greenwald & Kreiger, supra note 10, at 952.
18
Id., at 952-53 (describing the basic IAT technique).
5
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
when a black face or a good word appears. The participants then complete several trials
of this second task.
Researchers have consistently found that most white Americans find the second
task (in which they sort black and good from white and bad) more difficult than the first
(in which they sort white and good from black and bad).19 They make this determination
based on the amount of time it takes the participants to respond to the stimuli—the
“response latency.” Most white Americans produce higher response latencies when faced
with the stereotype-incongruent pairing (white-bad/black-good) than when faced with the
stereotype-congruent pairing (white-good/black-bad). In effect, most people express a
white preference. Researchers have determined that this white preference is not a product
of the order in which people undertake the tasks, their handedness, or some other
artifact.20 For most white adults, the stereotype-congruent associations make the task
easier and hence, speed reveals the presence of these associations.
The exact implications of this effect are a matter of some debate,21 but the
cognitive mechanisms underlying it are clear enough. The white preference arises from
well-established mnemonic links.
Whites more closely associate white faces with
positive words and black faces with negative words than the opposite. Thus, when they
complete the white-good versus black-bad trials, they need only make a judgment about
whether the stimulus that appears in the middle of the screen is positive or negative. The
incongruent association, in contrast, requires that they first judge whether the stimulus is
a word or a face and then decide on which side it belongs. Stereotype-incongruent
19
See Brian A. Nosek, Mahzarin R. Banaji & Anthony G. Greenwald, Harvesting Implicit Group Attitudes
and Beliefs from a Demonstration Website, 6 GROUP DYNAMICS 101, 105 (2002) (reporting data indicating
that white adults taking the IAT strongly favored the white-good versus black-bad pairing on the IAT).
20
Anthony G. Greenwald, Brian A. Nosek & Mahzarin R. Banaji, Understanding and Using the Implicit
Association Test: I. An Improved Scoring Algorithm, 85 J. PERSONALITY & SOC. PSYCHOL. 197 (2003).
21
Hal R. Arkes & Phillip E. Tetlock, Attributions of Implicit Prejudice, or, “Would Jesse Jackson ‘Fail’
the Implicit Association Test?,” 15 PSYCHOL. INQUIRY 257 (2004) (arguing that the IAT does not measure
bias or prejudice); Mahzarin R. Banaji, Brian A. Nosek & Anthony G. Greenwald, No Place for Nostalgia
in Science: A Response to Arkes and Tetlock, 15 PSYCHOL. INQUIRY 279 (2004) (responding to the
arguments of Arkes and Tetlock).
6
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
associations interfere with the sorting task in much the same way that the use of blue ink
can make the word “red” hard to read.22
Psychologists disagree about whether differences in response latencies on the IAT
amount to anything more than mnemonic associations. Some have argued that IAT
scores do not truly measure attitudes, biases, or deeply held beliefs, but measure
familiarity with cultural stereotypes.23 Most researchers, however, contend that the IAT
scores reveal the influence of unconscious bias.24 As evidence for their position, these
researchers point to the fact that African-Americans do not express a white preference,
even though they are apt to be at particularly aware of racial stereotypes. 25 Moreover,
IAT studies have been run in many countries with very different groups of subjects. 26
Indeed, the first published study using the IAT involved Koreans and Japanese, rather
than whites and African-Americans.27 Finally, some recently conducted research directly
refutes the cultural familiarity thesis.28
Whatever the appropriate interpretation and implications, evidence that the white
preference, and other invidious preferences, on the IAT is impressive. Hundreds of
thousands of people have participated in IAT research through the use a website
maintained by Mahzarin Banaji and others (www.projectimplicit.org).29 This website
documents the widespread presence of implicit associations toward women, the elderly,
homosexuals, and numerous racial and ethnic groups.30 Researchers from disciplines as
22
See John R. Stroop, Studies of Interference in Serial Verbal Reactions, 18 J. EXPERIMENTAL PSYCHOL.
643 (1935) (presenting evidence that words colored differently from their semantic meaning are difficult to
read).
23
See Arkes & Tetlock, supra note 211; Gregory Mitchell & Philip E. Tetlock, Antidiscrimination Law
and the Perils of Mindreading, 67 OHIO ST. L.J. 1023, 1023 (2006); Richard E. Redding, Bias or
Prejudice? The Politics of Research on Racial Prejudice, 155 PSYCHOL. INQUIRY 289 (2004).
24
See Greenwald, Nosek & Banaji, supra note 21; Samuel R. Bagenstos, Implicit Bias, “Science”, and
Antidicrimination Law (unpublished manuscript on file with authors).
25
See Nosek et al., supra note 19, at 105 (reporting data indicating that African-Americans show only a
weak white preference on the IAT).
26
Greenwald & Kreiger, supra note 9, at 956-58 (describing the range of IAT studies that have been
conducted).
27
Anthony G. Greenwald, Debbie E. McGhee & Jordan L. K. Schwartz, Measuring Individual Differences
in Implicit Cognition: The Implicit Association Test, 74 J. PERSONALITY & SOC. PSYCHOL. 6 (1998).
28
Eric L. Uhlmann, T. Andrew Poehlman, & Brian A. Nosek, Automatic Associations: Personal Attitudes
or Cultural Knowledge? (Unpublished Manuscript, 2007).
29
Greenwald & Kreiger, supra note 4, at 956.
30
Id.
7
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
diverse as political science and marketing31 have used the IAT in their research because it
enables them to identify attitudes that individuals might not know they possess or that
they might not want to share with the researcher.32
The IAT is not the only methodology that psychologists have devised to assess
the influence of unconscious stereotypes. Many studies use subliminal “priming” as a
means of studying implicit associations.33 In a typical priming study, researchers present
stimuli (such as words or face) to subjects so rapidly that subjects cannot identify the
stimuli being presented.
Nevertheless, presenting African-American faces or words
associated with African Americans in this subliminal fashion influences people’s
judgment of subsequent stimuli. In one study, for example, police and parole officers
exposed to words associated with African Americans subsequently judged a juvenile
offender (whose race was not identified) more harshly than when they had been exposed
to race-neutral words.34 Studies like this suggest that for many people, words associated
with African Americans trigger negative associations or stereotypes which then affect
their judgment.
B. Implicit Bias and Behavior
Even if implicit bias is as widespread as the IAT studies suggest, it does not
necessarily lead to racially disparate treatment. Conversely, the existence of widespread
racial disparities in society is not, in and of itself, a demonstration of the influence of
implicit bias. Only if researchers can show that implicit biases influence decision makers
of interest can we infer that implicit bias is a cause of racial disparities. More to the point
of our study, are judges who harbor negative associations towards African Americans apt
to treat African-American defendants differently than white defendants?
31
Greenwald et al., supra note 20.
See Anthony G. Greenwald, Mahzarin R. Banaji, Laurie A. Rudman, Shelly D. Farnham, Brian A.
Nosek, & Deborah S. Mellott, A Unified Theory of Implicit Attitudes, Stereotypes, Self-Esteem and SelfConcept, 109 PSYCHOL. REV. 3, 8 (2002).
33
John A. Bargh, The Four Horsemen of Automaticity: Awareness, Intention, Efficiency, and Control in
Social Cognition, in HANDBOOK OF SOCIAL COGNITION 1-40 (R. Wyer, & T. Srull eds., Lawrence Erlbaum
Associates, Inc. 1994).
34
Sandra Graham & Brian S. Lowry, Priming Unconscious Racial Stereotypes About Adolescent
Offenders, 28 L. & HUM. BEHAV., 483 (2004).
32
8
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
As a general matter, implicit bias, at least as measured by the IAT, appears to
predict behavior in some settings. In a recent review, Greenwald and his colleagues
identified over sixty studies assessing the relationship between various versions of the
IAT and observable behaviors.35 The review shows that some IAT scores correlate with
judgment, choices, physiological responses, non-verbal behavior, and attributions. These
reviewers found that across the full range of studies, they found that the average
correlation between implicit measures of attitudes and behaviors is 0.27.36 This average
masks enormous variation. Among the studies involving preferences for foods, brands,
or political parties, the correlation between the implicit measures and explicit preferences
is quite high—more like 0.5.37 This is unsurprising because there is often a very close
match between implicit associations with brands – for example, a more positive
association with Coca-Cola than with Pepsi – and the behavior at issue – choosing to
drink Coca-Cola over Pepsi.38
Positive or negative associations with a job applicant’s race, however, might have
a more complex relationship with a hiring decision.
Similarly, the overall average
correlation includes studies of some relationships that might be even more complex, such
as the correlation between implicit attitudes about romantic partners and decision to end
the relationship, and the implicit attitudes towards a person’s mother and spouse abuse.
The overall predictive validity of the IAT across domains might therefore either overstate
or understate its ability to predict racial discrimination in the legal system. Moreover, the
predictive ability of race IATs for legal judgments might be too broad an issue; decisions
involving criminal defendants might be more (or less) related to IAT scores than are
decisions involving civil litigants. Even within the criminal process, decisions involving
35
Anthony G. Greenwald, T. Andrew Poehlman., Eric L. Uhlmann & Mahzarin R. Banaji, Understanding
and Using the Implicit Association Test: III. Meta-Analysis of Predictive Validity. – J. PERSONALITY &
SOC. PSYCHOL. -- (forthcoming, 2008).
36
Id at __.
37
Id at __.
38
See Dominika Maison, Anthony G. Greenwald, & Ralph H. Bruin, Predictive Validity of the Implicit
Association Test in Studies of Brands, Consumer Attitudes, and Behavior, 14 J. CONSUMER PSYCHOL. 405,
410-2 (2004). See also Gregory S. Parks & Jeffrey J. Rachlinski, Unconscious Bias and the 2008
Presidential Election (Unpublished Manuscript on file with authors) (describing how unconscious bias
affects evaluations of political candidates).
9
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
bail might have a different relationship with IAT scores than decisions involving prison
sentences.
Recognizing the intense interest in the IAT as a potential measure of unconscious
racism, Greenwald and his co-authors also reported the predictive power of the IAT
among the studies in their review that measure stereotypes separately from the studies on
other topics.39
Among the 61 studies included in the review, 24 concerned the
relationship between implicit attitudes towards one or another protected status and
behavior. Of these studies, 19 involved implicit attitudes concerning race or ethnicity;
three involved implicit attitudes concerning gender; and two involved implicit attitudes
concerning sexual orientation.40 The correlation between the implicit measures and the
observed behaviors averages 0.25 among these studies. Putting aside which behavior
researchers tried to predict with IAT scores, this figure puts implicit measures of
stereotypes as somewhat less predictive of “behavior” than measures of consumer
preferences and somewhat more predictive than measures of personal relationships. The
0.25 correlation means that the implicit measures of stereotype can prediction roughly
6% of the variation in actual behavior.
This modest 6% figure, however, might understate the importance of the
relationship, or mask much larger effects. In a highly competitive employment market,
for example, a 6% difference would have an enormous impact on aggregate hiring
patterns. Because the data on the IAT suggests that the vast majority of white employers
express moderate to strong associations between white and good, most white applicants
will appear slightly (well, 6%) more appealing to employers than similarly qualified
African-Americans. Given large numbers of employment decisions, and large numbers
of applicants from which to choose, white employers favoring white applicants, even to a
small degree, will consistently choose white candidates, or pay a premium for white
candidates.
39
Greenwald et al., supra note 35, at __.
Note that some of the papers Greenwald and his co-authors include in their analysis report multiple
studies using independent samples of subjects. Counting all independent samples as different studies
therefore produces a somewhat different count: 27 studies of race or ethnicity; 4 studies of gender; and 3
studies of sexual orientation. Note that Greenwald et al report only 32 independent studies, whereas I count
34 among the studies they reported using in their appendix. I am unsure as to the source of this
discrepancy, but the paper remains in press as of this writing. Id.
40
10
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Furthermore, the effects of invidious unconscious associations can snowball.41
The 6% figure might mean that a whole range of intermediate evaluations favor a white
applicant. A white employer influenced by implicit associations might rate AfricanAmerican applicants as 6% less friendly in interviews, assess their paper qualifications
6% less favorably, and make 6% more negative inferences from comments by references.
Taken together, the aggregate effect of these negative inferences could produce a much
greater effect when the ultimate decision is made based on the product of an aggregation
of smaller judgments, each of which are affected by unconscious associations.
On the other hand, 6% might overestimate the effect of implicit bias on decision
making.
To the extent that factors not vulnerable to the influence of unconscious
associations are important to the decision-making process, they might trump the influence
of the implicit associations. For example, a hiring process that relies heavily on a score
on a written examination might reduce or eliminate the influence of any implicit biases.
Such tests might introduce other problems,42 but can reduce the influence of implicit bias
on the ultimate hiring decision.
Moreover, in at least some contexts, decision makers have shown an ability to
compensate for the effects of implicit bias.43 As a general matter, if properly motivated
or internally driven to suppress their own biases, people can produce judgments that
avoid the influence of stereotypes and bias.44 This holds true even if the source of the
bias is implicit.45 At least one study, by Fazio and his colleagues, showed that people
41
Kang & Banaji, supra note 11, at 1073.
See Claude M. Steele, A Threat in the Air: How Stereotypes Shape Intellectual Identity and
Performance, 52 AM. PSYCHOLOGIST 613 (1997) (describing how some features of standardized tests lead
to underperformance by African-Americans).
43
See Jack Glaser & Eric D. Knowles, Implicit Motivation to Control Prejudice, 44 J. EXPERIMENTAL SOC.
PSYCHOL. 164 (2008) (reviewing this research).
44
See Bridget C. Dunton, & Russell H. Fazio, An Individual Difference Measure of Motivation to Control
Prejudiced Reactions, 23 PERSONALITY & SOC. PSYCHOL. BULL. 316 (1997); E. Ashby Plant & Patricia G.
Devine, Internal and External Motivation to Respond Without Prejudice, 75 J. PERSONALITY & SOC.
PSYCHOL. 811 (1998).
45
See John F. Dovidio, Kerry Kawakami, Craig Johnson, Brenda Johnson, & Adaiah Howard, On the
Nature of Prejudice: Automatic and Controlled Processes, 33 J. EXPERIMENTAL SOC. PSYCHOL. 510
(1997); John A. Bargh, The Cognitive Monster: The Case Against the Controllability of Automatic
Stereotype Effects, in DUAL-PROCESS THEORIES IN SOCIAL PSYCHOLOGY 361 (S. Chaiken & Y. Trope eds.,
Guilford Press 1999); Patricia G. Devine, E. Ashby Plant, David M. Amodio, Eddie Harmon-Jones, & S. L.
Vance, The Regulation of Explicit and Implicit Race Bias: The Role of Motivations to Respond Without
Prejudice, 82 J. PERSONALITY & SOC. PSYCHOL. 835 (2002).
42
11
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
who both hold implicit biases and who are highly motivated to avoid the appearance of
prejudice, in fact, can easily overcompensate—treating African-Americans more
favorably than whites.46
Perhaps the most dramatic example of the relationship between implicit biases,
behavior, and the desire to control bias comes from a study by Jack Glaser, using the
“shooter bias” paradigm.47 The shooter bias refers to research using video-game style a
simulation in which a participant holds the game’s gun wile observing a video of a target
person pulling out either gun or an innocent object, like a wallet, out of his pocket.48
Most adults exhibit a “shooter bias” in that they are more likely to shoot AfricanAmerican targets than white targets—regardless of what object the target is pulling out of
his wallet.49 This effect correlates with having a white preference on the IAT.50 Glaser
has shown, however, that people who exhibit a white preference on the IAT, but who also
feel a strong internal need to control prejudice do not exhibit the shooter bias.
Context seems to inspire efforts to suppress bias as well. Sommers and Ellsworth,
for example, developed a scenario involving a criminal battery case that produced
assessments by white adults that varied with the race of the defendant.51
In their
scenario, 70% of white adults reading about a white defendant stated that they would
convict the defendant, while 90% of white adults reading about an African-American
defendant stated that they would convict. When they altered the scenario so that it
presented the crime as one involving a racially charged incident, however, the subjects
expressed no difference in conviction rates. Highlighting the role racial animus played in
the underlying incident induced people to adopt some corrective process that enabled
them to overcome their apparent bias.
46
Russell H. Fazio, Joni R. Jackson, Bridget C. Dunton, & Carol J. Williams, Variability in Automatic
Activation as an Unobtrusive Measure of Racial attitudes: A Bona fide Pipeline?, 69 J. PERSONALITY &
SOC. PSYCHOL. 1013 (1995).
47
Glaser & Knowles, supra note 43.
48
Joshua Correll, Bernadette Park, Charles M. Judd, & Bernd Wittenbrink, The Police Officer's Dilemma:
Using Ethnicity to Disambiguate Potentially Threatening Individuals, 83 J. PERSONALITY & SOC. PSYCHOL.
1314 (2002).
49
Id. at 1320.
50
Id. at 1321; Glaser & Knowles, supra note 43 at 168-69.
51
Samuel R. Sommers & Phoebe C. Ellsworth, White Juror Bias: An Investigation of Racial Prejudice
Against Black Defendants in the American Courtroom, 7 PSYCHOL. PUB. POL’Y & L. 201 (2001).
12
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Determining whether implicit bias predicts judgments is thus a complicated
business. Decision makers might rely on factors not subject to the influence of their
implicit biases or might be motivated to avoid the influence of these biases. Moreover,
most of the existing research connects implicit bias not to actual behavior, but to some
intermediary variable, such as non-verbal reactions, trait ratings, or neurological
responses. McConnoll and Leibold,52 for example, found that white subjects’ implicit
attitudes towards African Americans correlated with the non-verbal behavior they
expressed during an interview of an African-American job applicant, but they did not ask
the subjects to indicate whether they would hire the interviewee. Similarly, Rudman and
Lee found a correlation between implicit negative stereotypes about African Americans
and trait ratings of an African-American man, but they did not ask questions about how
the subjects would treat this individual.53 Thus, most of the studies identify, at most, an
indirect link between an implicit attitude and a target judgment of broad interest to
society.
Banaji, along with a group of medical doctors, has recently published an
impressive study that provides an initial response to the concern that IAT scores might
not predict judgment.54 These researchers measured implicit associations among a group
of internal medicine and emergency room physicians for white and black patients and
positive and negative words. They also asked the doctors to diagnose a patient based on
a description of symptoms and decide whether to offer that patient a treatment known as
thombolysis. Half of the patients were described as black and half as white. The
researchers found that that the greater the white preference that the doctors exhibited on
the IAT, the greater the likelihood that they would offer thrombolysis to white patients
but not black patients whom they had diagnosed as suffering from coronary artery
disease.
These results suggests that even among professionals trained to look at
52
Allen R. McConnell & Jill M. Leibold, Relations Among the Implicit Association Test, Discriminatory
Behavior, and Explicit Measures of Racial Attitudes, 37 J. EXPERIMENTAL SOC. PSYCHOL. 435 (2001).
53
Laurie A. Rudman & Matthew R. Lee, Implicit and Explicit Consequences of Exposure to Violent and
Misogynous Rap Music, 5 GROUP PROCESSES & INTERGROUP RELATIONS, 133 (2002).
54
Alexander R. Green, Dana R. Carney, Daniel J. Pallin, Long H. Ngo, Kristal L. Raymond, Lisa I.
Iezzoni, & Mahzarin R. Banaji, Implicit Bias Among Physicians and its Prediction of Thrombolysis
Decisions for Black and White Patients, 22 J. GEN. INTERNAL MED. 1231 (2007).
13
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
supposedly objective factors unrelated to race, unconscious associations concerning
African American and whites might affect judgment.
As important as this study is, it raises as many questions as it answers. The
researchers also found that white doctors who express white preferences ion the IAT
were more likely to diagnose black patients than white patients as having coronary artery
disease, based upon the same symptoms. Indeed, the doctors offered thrombolysis to an
equal number of black patients as white patients! As the authors rightly point out, this
does not mean there was no disparity; among patients who were diagnosed as suffering
from coronary artery disease, black patients were less likely to be offered the appropriate
treatment. It is at least curious, however, that doctors with implicit white preferences
would be more likely to diagnose coronary artery disease for black patients than white
patients, but less likely to treat it. The diagnosis disparity runs in the opposite direction
of the treatment-for-diagnosis disparity, and ultimately, the two effects actually cancel
each other out. Of course, if doctors behaved the same way in the real world, black and
white patients who presented the same symptoms would be treated in the same way. Thus,
though the IAT predicted a discriminatory act, implicit bias does not seem to result in
discrimination. Moreover, this finding of no overall discrimination is at odds with widely
reported data from the real world.
The authors of the study also asked participants whether they were aware of the
nature of the study. Of the 287 subjects who completed the study, 67 were excluded from
the main analysis based upon some awareness of the purpose of the study. The separate
analysis of the data from the aware subjects revealed a striking fact: Among physicians
who were aware of the purpose of the study, the physicians high in implicit bias were
more likely to recommend thrombolysis for black patients than were those who were low
in bias. The authors concluded that, “this suggests that implicit bias can be recognized
and modulated to counteract its effect on treatment.” Interestingly enough, fully sixty
percent of the subjects agreed prior to taking the IAT that “subconscious biases about
patients based on their race may affect the way I make decisions about their care without
my realizing it.” This generalized awareness of the risk of unconscious bias, unlike
14
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
awareness of the purpose of the study, did not seem to lead doctors to monitor their own
implicit biases in an effective way.
While intricate, this emerging body of research connecting IAT scores to behavior
can be summarized simply enough.
IAT scores predict at least some behaviors.
Although this research includes some dramatic decisions, such as medical diagnoses and
whether to shoot a potentially dangerous person, most predicted behaviors are really only
ancillary to important judgments, and include such as behavior trait ratings or eye
contact. Furthermore, the overall relationship between IAT score and behavior appears to
be a modest one, at best. People appear to be able to avoid making biased judgments if
they are sufficiently motivated to do so, either by an internal desire to avoid prejudice, or
if the situation they encounter appears to inspire them do so.
C. Implicit Bias and Legal Judgments
But what does this research mean for judgments in the legal system? Among the
dozens of studies assessing the external validity of the IAT, only two studies measure a
behavior direct interest to the legal system. In one of the two studies, conducted in
Germany, researchers correlated implicit attitudes towards native Germans and Turkish
immigrants among German college students with judgments of guilt of a Turkish
defendant. The researchers found a high correlation (0.58) between negative associations
with Turkish immigrants and judgments of guilt when the materials made “threatening”
aspects of the Turkish defendant salient.55 In the second study, researchers found that
American college students expressing a strong implicit bias in favor of whites relative to
Latinos gave longer sentences to Latino defendants than did those who did not show such
favoritism to whites.56
Thus, although researchers rightly conclude that the IAT can predict behavior in
some circumstances, scant evidence suggesting that implicit associations lead to disparate
55
Arnd Florack, Martin Scarabis & Herbert Bless, Der Einfluss Wahrgenommener Bedrohung auf die
Nutzung Automatischer Assoziationen bei der Personenbeurteilung /The Impact of Perceived Threat on the
Use of Automatic Associations in Person Judgments, 32 ZEITSCHRIFT FUER SOZIALPSYCHOLOGIE 249
(2001).
56
R.W. Livingston, When Motivation Isn’t Enough: Evidence of Unintentional Deliberative
Discrimination Under Conditions of Response Ambiguity (unpublished manuscript, 2002).
15
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
treatment of people by race in the American legal system exists. Only two studies that
use legal judgments as the target exist. Both use students as subjects, even though the
judgments addressed in the two studies are, in the real world, made by professionals.57
Furthermore, neither of the studies addressed white attitudes toward African-Americans;
indeed, as noted above, only one of the studies involved American study participants.
Before concluding that the evidence of widespread in-group favoritism and
stereotyping in implicit associations plays an important role in explaining racial outcomes
in the criminal justice system more evidence is needed.
Widespread variations in
outcome by race can be observed in the legal system,58 but no research yet links these
results with implicit attitudes, even indirectly. Do legal decisionmakers exhibit the same
pattern of implicit biases as do other Americans? Do any such biases influence their
judgments?
Can legal decisionmakers control the influence of their automatic
associations? In particular, we need to know whether the most critical guardians of
equality in the legal system—judges—have implicit racial biases, and if so, whether those
biases affect their judgment. We describe and report the results of just such a study in the
next Part.
We add, at the outset of our study, that we fully recognize that we have
emphasized disparities concerning African-Americans, rather than other races. We have
done this for three reasons. First, even though Latinos, Native Americans, and Asians are
also targets of racism, both explicit and implicit, some of the most striking disparities
involve African-Americans in the legal system. Second, the research on the IAT has
emphasized biases concerning African Americans as well. Third, our sample of judges
includes a large group of African-American judges, but few Latinos, few Asians, and no
Native Americans. We thus cannot draw any conclusions about the reactions of judges of
these ethnicities. Likewise, we have excluded gender form our discussion an analysis.
Although our sample includes a sizeable percentage of women, and we did, in fact,
57
The German study used a judgment of guilt as its target, which is conducted by jurors in the United
States, but in Germany, professional judges determine guilt.
58
David Baldus & George Woodworth, Race Discrimination and the Death Penalty in the Post Furman
Era: An Empirical and Legal Overview, with Recent Findings from Philadelphia, 83 CORNELL L. REV.
1638 (1998); See Mustard, supra note 3, at 295-300.
16
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
collect some data on women, our results were more ambiguous and require further study.
We therefore focus our attention here on biases involving African-Americans.
II. The Study
In our study, we sought to answer two research questions. First, do judges harbor
the same kinds of invidious associations found in most adults in the United States.
Second, if so, do these associations affect their judgment in familiar, legal contexts?
A. Summary of our Methods
To measure implicit associations involving race, we gave judges an IAT
measuring their associations between white faces or African-American faces and positive
or negative words. Because this particular IAT is so commonly used, we were able to
compare the judges’ results to the results of a large numbers of ordinary adults. The
judges in our study also made judgments on three hypothetical scenarios involving
criminal defendants in which the race of the defendants varied. In one scenario, we
explicitly identified the defendant’s race. In the other two scenarios we relied on a
subliminal priming process to suggest the race of the defendants. We reasoned that
judges’ implicit associations might affect how they would react to the criminal
defendants. Judges who show a white preference on the IAT might judge a criminal
defendant identified as African-American more harshly than a criminal defendant
identified as white. The reverse might be true for judges who show a black preference.
Thus, our methods enabled us not only to assess judges’ unconscious biases, but also the
relationship between these biases and their judgment of criminal defendants.
We also collected data on an IAT measuring judges’ associations between male
and female words with science and humanities words and on a hypothetical scenario of
an expert witness identified as either a male or female neurologist. We do not present
these results here, however, and confine this paper to the study of race.
17
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
B. The Judges
We recruited trial judges from three different groups to participate in our study.
In two cases, the judges were attending a small, annual judicial educational conference
held specifically for judges in their jurisdictions.59 At both of these conferences, we
presented a plenary session attended by most of the judges at the conference. The third
group consisted of judges attending an optional state-wide educational conference for
judges, who opted to attend a breakout session that we organized.60 All three groups of
judges have asked that we refrain from identifying their jurisdiction in any publication.61
We can, however, roughly identify the characteristics of these groups of judges.
The judges from the two annual conferences comprise the principal trial-court
judges in their jurisdiction. Both jurisdictions are large urban centers in the United
States: one in the Eastern half of the country and the other one in the Western half. The
Eastern jurisdiction is an area that includes a much higher percentage of AfricanAmericans than the Western jurisdiction. The judges from the East are appointed with
the assistance of a merit-selection panel for renewable fixed terms. The judges from the
West are appointed initially, but stand for election thereafter. In both jurisdictions, the
59
The sample of Eastern judges includes over 90% of the trial judges in their jurisdiction. Attendance was
lower among the Western judges; the sample includes roughly half of the judges in their jurisdiction.
These judges’ willingness to participate in our study was thus unlikely to have been affected by their
interest (or lack thereof) in the content of the material. In fact, the judges were not aware of the subject
matter of the talk before the session began. This was not our first presentation to the Eastern judges. Three
years earlier, we had presented a completely different set of materials to the same educational conference.
Some of the results from that earlier session have been published, also without identifying the jurisdiction.
(Andrew J. Wistrich, Chris Guthrie & Jeffrey J. Rachlinski, Can Judges Ignore Inadmissible Information:
The Difficulty of Deliberately Disregarding, 153 U. PENN. L. REV. 1251 (2005) (Reported in LEGAL
AFFAIRS, July/August, 2005)). Many of the judges were therefore familiar with our methods, although the
present study differs from our earlier work. Our prior work dealt largely with judicial reliance on heuristics
in making judgments, whereas this research is entirely devoted to the influence of race and gender on
judgment. This was our first presentation to the Western judges.
60
This group of judges thus self-selected both for attending the annual conference and for participating in
our research.
61
Their concerns might be justified. Some of our previous work has been reported in the New York Times
and the American Bar Association Journal, among other places. Patricia Cohen, Judges’ Reasoning is All
too Human, NEW YORK TIMES, June 30, 2001, at A17; Debra Casens Weiss, Judges Flunk Math Story Test,
ABA JOURNAL, Feb 19, 2008. The latter report leads with the unfortunate headline “Judges Flunk Math
Story Test”, which casts the judges in a more negative light than the data warrant. Interest in the present
paper is sufficiently high, that despite our own efforts to limit its use before it was final, it was cited by
Judge Jack Weinstein in a published opinion, United States v. Taveras, 424 F. Supp. 2d 446, 462 (E.D.N.Y.
2006), and discussed at length in a recent volume of the Annual Review of Law and Social Science. Lane et
al., supra note 10, at 441-42.
18
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
judges rotate through multiple departments within their jurisdictions. The judges from
the optional educational conference consisted of trial judges from throughout the state in
which the conference was held. These judges have an appointment process and serve in a
manner similar to that of the judges in the Western jurisdiction.
Not all of the judges who participated in our survey provided us with data that we
could use in our analysis. Among the 80 judges in the Eastern jurisdiction who attended
our session, one requested that we exclude his or her data (see below) and nine did not
provide us with demographic information.62 None of the 48 judges in the Western
jurisdiction who attended our session asked that we exclude their data. One of the judges
in the Western group did not provide demographic information and data from two of
these judges were lost during the session due to a computer malfunction. We excluded
the ten judges who did not provide demographic information from all of our analyses.
All 18 judges in the third group provided usable data. This left us with data from 133
trial judges: 70 from the Eastern jurisdiction; 45 from the Western jurisdiction; and 18
from the regional conference.
We did not ask the judges to identify themselves by name, but we did ask them to
identify their race, gender, exact title, general political orientation (Republican or
Democratic), and number of years of experience on the bench. 63
Table 1, below,
summarizes the demographic information that the judges provided.
62
We believe that this was largely accidental. To complete the demographic page, the judges had to
return to the written materials after completing the final IAT. Many judges failed to do so. We did not
realize this was an issue at our presentation in the Eastern jurisdiction, and hence we did not obtain this
data. In the subsequent presentations, we made sure that the judges completed this last page as we
collected the surveys.
63
We describe the questions precisely below in the Appendix.
19
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Table 1: Demographic information of the judges (% within group and number)
Demographic
Eastern
Western
Regional
Parameter
Jurisdiction (70) Jurisdiction (45) Conference (18) Overall (133)
White
52.9 (37)
80.0 (36)
66.7 (12)
63.9 (85)
Race
Black
42.9 (30)
4.4 (2)
5.6 (1)
24.8 (33)
Latino
4.3 (3)
11.1 (5)
16.7 (3)
8.3 (11)
Asian
0.0 (0)
4.4 (2)
11.1 (2)
3.0 (4)
Gender
Male
55.7 (39)
66.7 (30)
50.0 (9)
58.7 (78)
Female
44.3 (31)
33.3 (15)
50.0 (9)
41.4 (55)
Political
Democrat
86.6 (58)
64.4 (29)
64.7 (11)
76.0 (98)
Affiliation Republican
13.4 (9)
35.6 (16)
35.3 (7)
24.0 (31)
Average Years
9.8
10.8
9.3
10.1
of Experience
C. Methods and Materials
Our study involved a combination of computer tasks and responses to a set of
hypothetical scenarios on paper.
Because we did not ask the judges to identify
themselves, all responses were anonymous.
We also informed the judges that
participation in the survey was entirely voluntary. The final page of the questionnaires
gave the judges the opportunity to limit the use of their answers to the educational
workshop, thereby excluding them from discussion in other contexts and from use in any
publication. As we noted above, one Eastern judge exercised this option and this judge’s
results are not included in our analysis.
The study began with one of the computer tasks.64 As judges returned to the
conference room from a break, they found a laptop computer and a multi-page
questionnaire on the table in front of each seat. Both the computer screen and the title
page to the questionnaire identified the name of the conference, city, and date, and
instructed the judges not to begin until asked to do so.65
64
The computer tasks were all conducted on laptop computers rented for the purpose of running the
experiment. They were all relatively contemporary machines of similar makes. At the Eastern and
Western sessions, all were Hewlett-Packard NX 9010, at the regional conference, they were IBM
Thinkpads. All had 15" screens. The software to run the tasks was designed with a program called
“Inquisit v2.0”, created specifically for measuring implicit associations by a company called “Millisecond
Software.”
65
The instructions on the survey were as follows:
20
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
At the beginning of the session, one of us began by announcing that “today, we
shall ask you to participate in your own education.”66 We instructed the judges to
complete the computer tasks and surveys in order. We also told them that we would
compile the result of the session and present them at the end of the session. We also
reassured the judges that their results were anonymous, and that we had no way of
identifying them. Finally, we asked them to begin with the first computer task on the
laptop.
1. The First Computer Task: Subliminal Priming
Once the judges began the session, the computer instructed each judges that “This
exercise will involve a computer task followed by a series of written materials, and will
conclude with another computer task.
The computer tasks that follow might seem
unusual, but we shall explain their purpose during the discussion.” After
the
judges
continued, the instructions for the first computer task appeared.
The first computer task consisted of a subliminal priming program designed to
replicate a study conducted by Graham and Lowry.67 These researchers found that
showing police and parole officers words associated with African-Americans led these
officers to make more harsh judgments of juvenile offenders than showing them words
that have no clear association. The officers who had seen the African-American words
deemed the defendants to be more culpable, concluded that they had worse characters,
predicted that they would be more likely to recidivate, and recommended deserved a
Many of the points to be discussed at this session are best experienced directly. We therefore ask
that before the session starts, you participate in a series of exercises on the laptop computer and
evaluate a series of hypothetical cases in the pages that follow. (Participation in all aspects of this
exercise is voluntary, of course.) Please do not discuss these materials while you are participating.
We shall collect these surveys before the discussion and present the results during the session.
The first part of the exercise consists of a computer task. Please do not begin the task or turn
this page until asked to do so.
The instructions on the computer screen were:
JURISDICTION: Judicial Education Conference, DATE
We shall begin by making announcements as to the nature of this exercise.
Please DO NOT BEGIN until after the announcements.
After the announcements, please press the space bar to begin.
66
Judge Wistrich conducted the introduction at the Eastern and Western conferences; Rachlinski did it at
the regional conference.
67
Graham & Lowry, supra note 34.
21
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
harsher punishment.68 This effect occurred even though the researchers presented the
words subliminally. When asked to “recall” the race of the defendants, however, (the
race was not stated in the problem) the officers in both conditions generally stated that
they did not know, or that the defendant was white.69 Even though the participants were
not aware of the influence of the subliminal presentation, it affected their judgment.
Our first computer task used the same format as that in the study by Graham and
Lowry.
We asked the judges to undertake what appeared to be a simple spatial
recognition task on the computers.70 The task focused the judges’ attention on the center
of the screen while subliminally exposing judges to a series of words. Each word
appeared on the screen, off in one of the four corners, for 153 milliseconds before being
masked by a string of random letters.71 At that speed, words are extremely difficult to
68
Id. at 493, 496. Only Police officers predicted that the defendant was more likely to recidivate; parole
officers did not show any differences on this question. Id. at 493, 496.
69
The parole officers were twice as likely to misremember that the suspect was black than white after
being shown the African-American words than the neutral words, but police officers showed no
differences. Id. at 492.
70
At the beginning of the task, three asterisks appeared in the center of the screen. A sixteen-character
letter string then appeared in one of the four quadrants of the screen. The judges were instructed to press a
specific key on the left-hand side of the computer (the “E” key, which was marked with a red dot) when the
letter string appeared in one of the quadrants on the left and to press a specific key on the right-hand side of
the computer (the “I” key, which was also marked with a red dot) when a word appeared in one of the two
quadrants on the right. Reminders as to which key to press also remained on the computer screen
throughout the first task (i.e., “Press the “E” key for left” and “press the “I” key for right”). When the
judges identified the quadrant correctly, the word “CORRECT” would appear in the center in letters.
When the judges made an error, the word “ERROR” would appear instead. In either case, the three
asterisks would then replace the words “correct” or “error” and the task would repeat.
The exact instructions the judges saw are below:
Once you begin the first computer task, the screen will go blank, then three asterisks (* * *) will
appear in the center. Focus your attention on these. A string of letters will then appear in the
upper-right, lower-right, upper-left, or lower-left portion of the computer screen.
If the string appears on the left-hand side (either up or down), press the “E” key.
If the string appears on the right-hand side (either up or down), press the “I” key.
If you correctly identify the position, the screen will flash the word "correct"; if you identify the
wrong position, the screen will flash the word "error."
The task will then repeat a number of times. Other words may appear with the letter string.
Ignore these and try to identify the position of the letters as quickly as possible.
When you are ready, press the space bar to begin the task.
71
Each trial thus proceeded as follows: the three asterisks would appear in the center of the screen; 1200
milliseconds later (1.2 seconds) one of the prime words (selected at random) would appear in one of the
four quadrants (at random as determined by the computer); 153 milliseconds after that, the letter-strong
would appear over the prime; this would remain until the judge pressed either the “E” or “I” key; then
either the “correct” or “error” in the center (depending upon the judge’s response) and would remain for
roughly 1 second; then the three asterisks would replace the word “correct” or “error”; and the process
22
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
process consciously.72 The task repeated so that each judge experienced 60 exposures to
the words. Half of the judges saw words associated with African Americans, 73 and half
saw words with no common theme.74 After the 60th trial, the task stopped.75 The
computer screen then instructed the judges to turn to the written materials.76
2. The Written Materials Part 1: Juvenile Offenders
The written materials consisted of four one-page hypothetical cases, the third of
which we do not report here, as it dealt with gender. We asked judges to read the
scenarios and answer questions similar to those on which they found differences. Each
briefly described a set of facts, followed by a few questions.
The first two pages described cases of juvenile defendants, without identifying
their race. The first scenario, labeled “Shoplifter Case”, described the case of William, a
would repeat.
Due to an error in the computer programming, the judges in the Eastern conference were only
exposed to the subliminal prime for 64ms, rather than 153ms.
72
Graham and Lowry reported that none of the officers in their study was able to identify the nature of the
words being shown to them. Id. at 491. We did not ask our judges their assessment of what the words
were.
73
The words came directly from the Graham and Lowry study: graffiti, Harlem, homeboy, jerricurl,
minority, mulatto, negro, rap, segregation, basketball, black, Cosby, gospel, hood, Jamaica, roots, afro,
Oprah, Islam, Haiti, pimp, dreadlocks, plantation, slum, Tyson, welfare, athlete, ghetto, calypso, reggae,
rhythm, soul.
74
These words also came directly form Graham and Lowry: baby, enjoyment, heaven, kindness, summer,
sunset, truth, playful, accident, coffin, devil, funeral, horror, mosquito, stress, toothache, warmth, trust,
sunrise, rainbow, pleasure, paradise, laughter, birthday, virus, paralysis, loneliness, jealousy, hell,
execution, death, agony. Graham and Lowry used words that matched the African-American words for
positive or negative associations. Id. at 489.
75
Our study differed from that of Graham and Lowry in several ways, any of which might have affected
the results. First, Graham and Lowry used 80 trials, rather than the 60 we used. Second, because we ran a
large group of judges at the same time, we did not use audible beeps to indicate correct responses. Third,
our hypothetical defendants differed. We did not have access to the original materials Graham and Lowry
used (the authors declined to provide us with these materials), and so wrote our own. Fourth, we asked
fewer questions concerning the hypothetical defendants. Although we do not see how any of these
differences would necessarily affect the results, priming tasks can be sensitive to details.
76
The following appeared on the screen
Thank you for completing the first computer task.
Now please turn to the written materials.
Please leave this computer on with the screen up.
After you have completed four pages of written materials, please press the space bar to continue
with the final computer tasks.
In case a judge accidentally or mistakenly hit the space bar, we added another intervening page before the
second computer task, which appeared once the space bar was pressed. It read as follows:
If you have completed the four case summaries, please press the space bar to begin the final
computer task.
23
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
13-year old who had been arrested for shoplifting several toys from a large, upscale toy
store.77 The materials indicated that he had no prior record. They described some of the
details of his arrest by a security guard at the store. They provide some conflicting
evidence on the degree to which the defendant resisted arrest, but indicate no real dispute
over the fact that he was shoplifting or any dispute as to what he had been caught trying
to steal.78
We then asked the judges three questions about “William”.
First, “In your
opinion, without regard to the options actually available in this kind of situation, what
would be the most appropriate disposition of this case?” The materials listed seven
options under the question, arrayed from the least to most serious option, ranging from a
dismissal to a transfer to adult court.79 Second, we asked judges, “In your opinion, on a
scale of one seven, how likely is it that William will later commit a crime similar to the
one with which he is charged?” Finally, we asked, “In your opinion, on a scale of one to
seven, how likely is it that William will commit more serious crimes in the future?” The
last two questions provided a scale running from one to seven, with the one labeled “very
likely” and the seven labeled “not at all likely”.
The second page, labeled “Robbery Case”, described the case of Michael S, who
was arrested for armed robbery of a gas station when he was two days shy of his
seventeenth birthday.80 The materials indicated that he had one prior arrest for a fight in
the school lunchroom the previous year. They then described the crime, in which the
perpetrator threatened the clerk of a convenience store with a gun, and made off with
$267 in cash. Michael admitted the crime, claiming that his friends had dared him to do
it. After the fact pattern, we asked the same three questions as for the first scenario.
77
The location of the crime would reveal the jurisdiction and hence we delete it. The location was an
upscale shopping district.
78
The exact materials for this scenario and all others are included as Appendix A.
79
The options were as follows: 1) dismiss it with an oral warning; 2) adjourn the case in contemplation of
dismissal (assuming William gets in no further trouble); 3) Put William on probation for six months or
less; 4) Put William on probation for more than six months; 5) Commit William to a juvenile detention
facility for six months or less; 6) Commit William to a juvenile detention facility for more than six months;
7) Transfer William to adult court.
80
The use of an armed robbery breaks somewhat with Graham and Lowry who had used two simple
property crimes. Graham & Lowry, supra note 34, at 490.
24
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
3. The Written Materials Part 2: Battery and Race
We took the final scenario directly from the study by Sommers and Ellsworth,
described above.81 In their study, these researchers presented white adults in an airport
with a one-page description of an assault committed by one member of a high-school
basketball team against another. The assault had resulted from a shouting match between
two players. The defendant asserted that he felt that he was in physical danger and was
merely defending himself. Given the facts, however, his claim for a self defense is weak.
For half of the participants, the materials identified the defendant as an “AfricanAmerican male” and the victim as a “Caucasian male”; for the other half, the materials
identified defendant as Caucasian and the victim as African-American. The materials
first identified the defendant and victim, provided a summary of the prosecution’s case,
provided a summary of the defense’s case, asked for a verdict, and then asked for an
assessment of the participant’s confidence in the verdict.82
On the page that followed this scenario, the materials instructed the judges to
return to the computer for the final computer tasks with the following instruction: “Please
return to the laptop computer to complete the computer tasks before completing the
remaining pages.”
4. The Second Computer Task: The IAT
After assessing the written scenarios, the judges returned to the computers to
complete an IAT designed to compare the relative strength of positive and negative
associations among the judges with African American and white faces.83 As with all
variations on the IAT, we measured the strength of judges’ associations by having them
perform two versions of the IAT task. The first required judges to sort stimuli either as
81
Sommers & Ellsworth, supra note 51. We thank the authors for graciously sending us the materials and
giving us permission to use them.
82
We used the same question to elicit verdicts and confidence ratings as Sommers and Ellsworth used:
“Based on the available evidence, if this were a bench trial, would you convict the defendant?” Below this
were the words “Yes” and “No”. Finally, we asked the judges, “How confident are you that your judgment
is correct?” Below this question, the materials presented a nine-point scale, with “1" labeled “Not at all
Confident” and “9" labeled “Very Confident”. Id. at 217.
83
The judges actually completed two IATs: the race IAT that we report here (which all judges completed
first), and a gender IAT measuring the strength of association between male and female with science of
liberal arts. We do not report the results of gender IAT here.
25
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
white faces and positive words, or as African-American faces and negative words. The
second required judges to sort stimuli either as white faces and negative words, or as
African-American faces and positive words. We took the differences in length of time it
took judges to complete each task, that is the response latency, as our measure of the
differences in association strength. This difference is the judges’ IAT score.
We introduced the remaining computer tasks to the judges with some general
instructions first. These (and all of our IAT instructions) were taken from the website for
“project implicit” and match the standard instructions given by IAT researchers. They
informed the judges that they would be sorting target stimuli that would appear in the
center of the screen into one of the four categories, with two categories paired on one side
of the screen and the other two categories paired on the other side of the screen.84
To assess the IAT score in judges, we followed the procedures recommended by
Greenwald, Nosek, and Banaji to maximize the reliability of our IAT measures. 85 These
authors recommend using seven rounds of IAT trials. Most of these rounds consist of
practice trials meant to give the judges some practice with the task and to reduce the
effects that the order of presentation might have. We use only two of these seven rounds
to obtain our measure of the judges’ IAT scores. We report the exact details of the IAT
procedure as Appendix B.
5. The Written Materials Part 5: Demographic Information
The final page of the materials requested demographic information from the
judges. We asked the judges to identify: their exact title (“What is the title of the judicial
position you currently hold?”); their experience (“How many years have you served as a
84
85
The exact instructions at the outset of the IAT were as follows:
The remaining computer tasks involve making CATEGORY JUDGMENTS.
Once the tasks begin, a word or words describing the CATEGORIES will appear in the upper left
and upper right corners of the computer screen.
A TARGET word or picture will also be displayed in the center of the screen, which you must
assign to one of the two categories
Please respond AS RAPIDLY AS POSSIBLE, but don't respond so fast that you make many
errors. (Occasional errors are okay.)
An "X" will appear when you make an error. Whenever the "X" appears, correct the mistake by
pressing the other key.
Greenwald, Nosek & Banaji, supra note 20.
26
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Judge (in any position)?”); their gender (“Please identify your gender:”); how they have
spent their judicial career (“During your judicial career, approximately what percentage
of your time has been devoted to the following areas: Criminal cases, Civil Cases, Family
law cases, Probate or Trusts, Other”); their political orientation (“Which of the two major
political parties in the United States most closely matches your own political beliefs?”);
and their race (“Please identify your race (Check all that apply): White (non-Hispanic);
Black or African-American; Hispanic or Latino; Asian; Native American or Pacific
Islander; Other”).
D. Scoring the IAT
We calculated the judges’ IAT scores in two different ways. First, for each judge,
we calculated the difference between the average latency in the stereotype-congruent
rounds in which the judges sorted white-good versus black-bad and the average latency in
the stereotype-incongruent rounds in which the judges sorted white-bad versus blackgood. This procedure follows the method that other researchers have used in reporting
data from hundreds of thousands of participants collected on the Internet.86 Hence, we
can compare this average score with that of large groups of ordinary adults.
In an exhaustive review of IAT methodology, however, Greenwald and his
colleagues concluded that the average difference might not be the best measure of
implicit associations.87 These researchers found that people who are slower on the task
produce larger differences in their IAT scores. This tendency confounds the IAT score,
as people who are simply less facile with a keyboard will appear to have stronger
stereotypic associations. Furthermore, Greenwald and his colleagues also found that the
average difference did not correlate as well with people’s decisions and behavior as other
scoring methods.88
After conducting their review, Greenwald and his colleagues
identified a preferred scoring method, which we followed to assess the correlation
between IAT effects and judges’ decisions.89 The method essentially uses the mean
difference for each participant divided by the standard deviation of that participant’s
86
87
88
89
Nosek et al., supra note 19.
Greenwald et al., supra note 20.
Id. at 203.
Id. at 214 (Table 4).
27
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
response latencies, although it includes some variations, which we describe in Appendix
C.
The choice of scoring method for the IAT requires making several judgments
about the data. It requires deciding which of the seven rounds to use (some studies make
use of the practice rounds); how to manage latencies that seem too long or too short; how
to assess erroneous responses; how to identify and score participants who respond too
slowly, too quickly, or make too many errors; whether to standardize the responses;
whether to use every round in a trial (or drop the first two, which commonly produce
excessively long latencies). Greenwald and his colleagues tested essentially all variations
on answers to these issues and produced a scoring method that they believe maximizes
the correlation between the IAT and observed behavior.90
Despite these complexities, our scoring method can be summarized quickly. We
subtracted the “average latency” in the stereotype-congruent round from the stereotypeincongruent round for each judge to facilitate a comparison between the judges and
ordinary adults. To correlate the IAT effects with the decisions that judges made in the
scenarios, however, we wanted a standardized measure. To obtain this standardized
measure, we divided the mean difference for each judge by the standard deviation of that
judge’s latencies in the target rounds.
Our scoring methods largely follow those
recommendations from Greenwald and his colleagues so as to give us the greatest chance
of identifying a relationship between the IAT measures and decisions on the scenarios.
The full account of our scoring methods is included as Appendix C.
III. Results
We present the results in several parts. First, we present the overall results from
the IAT and compare them to data from Internet surveys conducted by others. Second,
we present the results of our replication of Sommers & Ellsworth’s research on race and
criminal defendants. We also assess whether the race of the judge and score on the race90
Id.
28
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
IAT affected judgments of the black and white criminal defendants. Fourth, we present
the results of the subliminal priming task and its connection with the judges’ IAT
scores.91
A. IAT Scores
Table 2 reports the scores for the judges on the IAT. The results revealed a strong
white preference among the white judges. The white judges performed trials on the
stereotype-congruent pairing (white and good against black and bad) 216 milliseconds
faster than trials in the stereotype-incongruent pairing (black and good against white and
bad). Among the 85 white judges, 74 (or 87.1%) showed a white preference on the IAT.
The African-American judges demonstrated no clear preference overall. These
judges performed trials on the stereotype-congruent pairing (white and good against
black and bad) a mere 26 milliseconds faster than trials in the stereotype-incongruent
pairing (black and good against white and bad). Furthermore, only 14 of the 43, or
44.2%, of the African-American judges showed a white preference. Comparing the mean
IAT scores of the white judges with those African-American judges revealed that the
white judges expressed a significantly large white preference.92
91
We analyzed the three groups of judges separately, but there were no significant differences between the
judges, except as noted below, so we have kept them together throughout the analysis. Similarly, we found
no differences between the judges on the basis of the gender, political affiliation, or experience. Because
previous research on the IAT suggests that Latinos score somewhat closer to African-Americans on the
IAT we used, we combined the few Latino judges with the African-American judges for these analyses.
Nosek et al., supra note 19, at 110 (Table 2). Similarly, we combined the Asian judges with the white
judges.
92
t(82) = 4.94, p <.0001. Throughout this paper, we reserve the use of the words “significant” and
“significantly” for statistical significance.
29
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Table 2: Results of Race IAT by Race of Judge
Race of
Mean IAT Score in milliseconds
Judge
(and standard deviation)*
(sample size)
Judges
Internet Sample93
Percent of Judges
with lower average latencies
on the white/good versus
black/bad round
White (85)
216 (201)
158 (224)
87.1
Black (43)
26 (208)
39 (244)
44.2
* Positive numbers indicate lower latencies on the white/good versus black/bad round)
The judges in our sample produced results on the IAT that are similar to those of
ordinary adults. The white judges expressed a significantly higher effect than observed in
the sample of white subjects obtained on the Internet.94 This difference, however, might
not mean that the white judges really harbor more intense white preferences than the
general population. We did not vary the order in which we presented the materials and
the judges were overall much slower than the other adults with whom we are making this
comparison.95 Both of these factors would artificially inflate the judges’ IAT scores
relative to those of white adults in the Internet sample. The African-American judges did
not produce significantly different IAT scores than those observed in the sample of
African-American subjects obtained on the Internet.96
The judges themselves varied somewhat in their IAT scores. White judges in the
Eastern jurisdiction expressed an average standardized preference of 0.33, compared to
0.48 and 0.55 in the Western jurisdiction and the statewide conferences, respectively.
93
The Internet sample is taken from Nosek et al., supra note 19, at 105.
t(84) = 2.26, p = 0.026. We compared our results to those of the Internet sample reported in Nosek et
al., supra note 19, at 105. In making this comparison, we took the effect size among Internet sample of
0.83 standard deviations to be the “population” effect size among white participants on the Internet, and
tested whether our observed difference, with our observed standard deviation, would be likely to be reliably
higher or lower than the effect in the Internet data.
95
See Appendix C. Our procedures, however, minimize the effects of order of presentation. Greenwald et
al., supra note 20, at 210 (Table 2)report that the effect of order of presentation is less than 1%, using the
methods we followed.
96
t(42) = 0.18, p = .86. In conducting this test, we took the effect size among Internet sample of 0.16
standard deviations to be the “population” effect size among African-American participants on the Internet,
and tested whether our observed difference, with our observed standard deviation, would be likely to be
reliably higher or lower than the effect in the Internet data. The priming condition did not appear to affect
the judges’ IAT scores.
94
30
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
These differences were marginally significant.97 Because the African-American judges
in our study were concentrated largely in the Eastern jurisdiction, similar tests for
variations among these judges would not be reliable.
B. Race and the Battery Case
Table 3 reports the judges’ decisions in the battery case. The white judges were
equally willing to convict the African-American defendant and the white defendant. As
compared to the twenty percent higher conviction rate for African-American defendants
that Sommers and Ellsworth observed among white adults in their study, the white judges
behaved differently.98
Table 3:
Race of
Judge
Percentage of Judges Convicting the Defendant, by Race of Judge and
Race of Defendant (and sample size)
Race of Defendant
White
Black
Total
White
79.5 (44)
73.3 (45)
76.4 (89)
Black
92.3 (26)
50.0 (18)
75.0 (44)
Total
84.2 (70)
66.7 (63)
75.9 (133)
In contrast, African-American judges expressed far more willingness to convict
the white defendant than the African-American defendant. All but two of the AfricanAmerican judges who evaluated the white defendant indicated that they would convict,
whereas only half of the African-American judges who evaluated the African-American
defendant indicated that that they would convict the defendant. Statistical analysis of the
conviction rates showed that the race of the defendant influenced the African-American
judges’ assessments, but not the white judges’ assessments.99 Analysis of the judges’
assessments of their confidence in their verdicts produced similar results.100
97
F(2, 86) = 2.79, p = .07.
The difference is significant χ2(1) = 6.74, p < .01 (using the expected conviction rates of 70% and 90%
for white and African-American defendants, as reported by Sommers & Ellsworth, supra note 51, at 217).
99
The analysis consisted of a logistic regression of the verdict against the race of the defendant, the race of
the judge and the interaction of these two parameters. The interaction was significant (z = 2.12, p = .03),
which was the result of the differential treatment of the two defendants by the African-American judges.
98
31
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
We also assessed the relationship between the judges’ IAT scores and their
judgment of the defendant. This analysis consisted of a logistic regression of the verdict
against the race of the defendant, judges’ IAT scores, and an interaction of these two
variables. A significant effect of the interaction would have indicated that the judges’
IAT scores affected how the race of the defendant affected their judgment. Although the
race of the defendant was significant,101 the interaction was not.102 Analysis of the
confidence ratings produced the same result.103
Because the white and black judges seemed to react differently to the problem, we
also conducted the analysis to account for these differences. To do this, we regressed the
judges’ verdicts against the judges’ race, the race of defendant, the judges’ IAT scores,
and all interactions between these variables. In effect, this analysis combines the last
two. This analysis produced several significant terms, which duplicated the results of the
first two analyses, reported above. 104 The three-way interaction between race of judge,
race of defendant and IAT score was significant.105
This result showed that the IAT
scores of the African-American and white judges had different effects on the judges’
The race of the defendant was also significant. z = 2.81, p = .005, indicating that overall, the judges were
less likely to convict the African-American defendant, than the white defendant.
100
We combined the 9-point confidence measure with the binary outcome to create an 18-point scale. In
our coding, a “1” corresponded to a judge who was very confident that the defendant should be acquitted,
whereas an “18” corresponded to a judge who was very confident that the defendant should be convicted.
The average confidence that the judges expressed in the defendant’s guilt were as follows: white judges
judging white defendants—13.64; white judges judging African-American defendants—12.2; AfricanAmerican judges judging white defendants—16.08; African-American judges judging African-American
defendants—9.89. Statistical analysis of these results (by ANOVA) produced results consistent wit the
analysis of the verdicts alone. That is, the judges were significantly more convinced of the white
defendant’s guilt than the African-American defendant (F(1, 129) = 15.04, p < .001) and that this disparity
was much more pronounced among African-American judges. F(1, 129) = 5.84, p < .025.
101
z = 2.54, p< .025.
102
z = 2.54, p< .025.
103
We also replicated this analysis with the 18-point confidence ratings. See infra note 100. Specifically,
we regressed the judges’ confidence in the defendant’s guilt against the defendant’s race, the judges’ IAT
score, and the interaction between the race and IAT score. As with the verdict itself, this analysis showed
that the race of the defendant was significant, t-ratio = 3.49, p< .001, but the interaction between race of
defendant and IAT score was not. t-ratio = 1.51, p = .13.
104
Both the race of the defendant and the interaction between race of judge and race of the defendant were
significant, just as they were in the simpler models. (race of defendant, z = 1.99, p = .05; interaction
between race of the judge and race of the defendant, z = 2.35, p = .02. The interaction of the defendant’s
race and IAT score was not significant. z = 1.00, p = .23.
105
z = 2.18, p = .03.
32
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
reactions to the race of the defendant. Analysis of the confidence ratings produced
similar results.106
To allow us to interpret this interaction, we ran the analysis separately for
African-American judges and for white judges. That is, we ran two logistic regressions
of the judges’ judgment of the defendant against the race of the defendant, the judges’
IAT scores, and an interaction of these two terms: one with just white judges and one
with just African-American judges. White judges showed no significant trends. The
tendency they expressed, however, was for judges with a greater association between
white and good to express a greater propensity to convict the white defendant relative to
the black defendant.107 African-American judges expressed a significant opposite trend;
those judges with higher IAT scores were more likely to convict the African-American
defendant relative to the white defendant.108 In effect, white judges tended to behave in a
manner opposite to our predictions, whereas African-American judges were consistent
with our predictions. An analysis of confidence ratings produced similar results.109
In sum, IAT score predicted nothing among the white judges. If anything, the
white judges who expressed greater white preferences scores were less likely to convict
the African-American defendant relative to the white defendant. Among the AfricanAmerican judges, however, a greater association between white and good produced a
greater propensity to convict the African-American defendant.
106
Regressing the 18-point confidence rating against the race of the judge, the race of the defendant, the
judges’ IAT scores, and all interactions between these variables revealed significant effects for race of the
defendant, t-ratio = 2.95, p = .005, a significant interaction of race of the defendant with race of the judge,
t-ratio = 2.68, p = .01 and the three-way interaction of race of judge, race of defendant and IAT score. t–
ratio = 2.68, p = .02. The interaction of race of defendant and IAT scores was still not significant in this
model. t–ratio = 1.27, p = .20.
107
z = 1.15, p =.25.
108
z = 1.87, p =.06. Given the high conviction rate of the African-American judges of the white
defendant, this trend actually meant that they were more likely to convict the African-American defendants
as their IAT scores increased.
109
The white judges expressed a tendency toward a greater propensity to convict the white defendant
relative to the African-American defendant as the IAT score increased, but the trend did not approach
significance. t-ratio = 1.00, p = .40 The African-American judges showed the opposite trend, which was
significant. t-ratio = 2.25, p =.03.
33
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
C. Priming
Table 4: Average Results on Juvenile Shoplifter (All 3 questions on a 7 point scale:
Higher numbers indicate harsher judgments*)
Race of
Judges
Prime (and n)
All
Judges**
Black
White
Black
Q1: Disposition Q2: recidivismsame crime
Q3:recidivismmore serious
crime
(63)
2.34
2.58
2.23
Neutral (70)
2.40
2.36
1.94
Black
(41)
2.46
2.76
2.42
Neutral (48)
2.48
2.50
2.02
Black
2.14
2.23
1.86
(22)
Neutral (22)
2.23
2.05
1.77
*Note: the 7-point scale for questions 2 and 3 have been transposed from the original for
this table, so that higher numbers consistently meant harsher judgment.
As Table 4 shows the subliminal priming task appeared to have little effect on the
judges’ assessments of the shoplifter, at least in the aggregate. Judges primed with the
African-American words did not produced significantly different ratings than the judges
primed with the neutral words on judgments as to the appropriate penalty, whether the
defendant was likely to commit the same crime again, and whether the defendant was
likely to commit a more serious crime did not vary.110
To isolate the potential effect of the judges’ IAT score on these judgments, we
conducted ordered logit regressions of the three dependent measures against the priming
condition, the judges’ IAT scores, and the interaction of these two variables. We found
that the interaction term had a marginally significant effect on the judges’ sentences, 111
meaning that the judges with different IAT scores responded differently to the subliminal
priming. Specifically, the more that the that judges showed a white preference on the
IAT, the harsher the sentence they endorsed after exposure to the African-American word
list than after exposure to the neutral word list. This effect occurred among judges
throughout the full range of IAT scores, meaning that judges who expressed a black
110
111
Question 1, z=0.51, p = .61; Question 2, z = 0.73, p = .46; Question 3, z =1.09, p = .28.
z = 1.84, p = .07.
34
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
preference on the IAT gave less severe sentences after exposure to the African-American
word list than after exposure to the neutral list. This effect was not significant in our
analysis of the two recidivism questions, although the trend was similar.112
Table 5: Average Results on Juvenile Armed Robber (All 3 questions on a 7 point
scale: Higher numbers indicate harsher judgments*)
Race of
Q1: Disposition Q2: recidivismQ3:recidivism-more
Judges
Prime (and n)
same crime
serious crime
All
Judges**
White
Black (63)
4.92
3.54
3.17
Neutral (70)
4.97
3.61
3.48
Black (41)
5.00
3.71
3.32
Neutral (48)
5.04
3.70
3.49
Black
Black (22)
4.77
3.23
3.32
Neutral (22)
4.81
3.41
3.45
*Note: the 7-point scale for questions 2 and 3 have been transposed from the original for
this table, so that higher numbers consistently meant harsher judgment.
As Table 5 shows that the African-American priming words had somewhat more
effect on the second task, although it produced some surprising results. The results
varied slightly by question, as the Table suggests.
The priming condition had no significant effect on the primary question of the
disposition of the offender.113 Neither did the priming condition produce a significant
effect on the second question (likelihood of committing a similar offense).114 The third
question (“how likely is it that Michael will commit more serious crimes in the future?”)
produced an effect that approached significance.115 The effect was, however, in the
opposite direction from what we had predicted—the judges made slightly harsher of the
defendant after exposure to the neutral prime.
112
For the first recidivism question, z = 1.41, p = .16. On the second recidivism question, z = 1.49, p = .14.
On these questions, the African-American judges and the white judges seemed to respond in similar ways.
We ran the full model (predictors of prime, judge race, IAT, and all interactions between these variables)
on all three variables as well. Adding the judge race terms and interactions did not produce any significant
effects.
113
z =.17, p = .87.
114
z = 0.09, p = 0.93.
115
z = 1.62, p = .11.
35
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
To isolate the potential effect of the judges’ IAT score on these judgments, we
conducted ordered logit regressions of the three dependent measures against the priming
condition, the judges’ IAT scores, and the interaction of these two variables. As above,
we observed a marginally significant interaction between the subliminal priming effect
and the IAT score on the sentence judges assigned to the defendant.116 As they did with
the shoplifter, to the extent that judges expressed a white preference on the IAT, they
gave more harsh sentences to the defendant after exposure to the African-American word
list; the opposite was true for judges who expressed a strong black-good associations. On
the two recidivism questions, this effect was not significant.117
IV. Discussion
The research supports three conclusions. First, it shows that judges carry the
same kinds of invidious implicit associations concerning race as do the rest of us.
Second, it shows that these associations can affect judges= judgments, at least in contexts
where they are unaware of a need to monitor their own reactions. Third, and conversely,
when judges are aware of a need to monitor their own responses for the influence of
automatic racial associations, they can do so effectively. Below, we elaborate on these
interpretations of our results and identify reforms the criminal justice system might
implement to negate the potential impact of invidious implicit associations in the
courtroom.
A. Interpretation
The first aspect of this portrait of the unconscious bias in judges was the most
predictable. Given the large number of Americans who have taken the IAT that we used,
and the frequency with which white Americans display at least a moderate automatic
116
z = 1.85, p = .06.
For the first recidivism question, z = 0.62, p = .53; on the second recidivism question, z = 0.54, p = .59.
As above, on these questions, the African-American judges and the white judges seemed to respond in
similar ways. We ran the full model (predictors of prime, judge race, IAT, and all interactions between
these variables) on all three variables as well. Adding the judge race terms and interactions did not produce
any significant effects.
117
36
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
preference for white over black, it would have been surprising if white judges had failed
to exhibit the same automatic preferences.
It is worth noting, however, that the research on “chronic egalitarians” suggests
that this result was not inevitable. Some whites with longstanding and intense personal
commitments to eradicating bias in themselves—so called “chronic egalitarians”—do not
exhibit the preference for whites over blacks on the IAT that most white adults show.118
Judges, even with their professional commitment to the equal application of the law, do
not appear to develop the same habits of mind as the chronic egalitarians. The proportion
of white judges in our study who revealed automatic associations of white with good and
black with bad was, if anything, slightly higher than the proportion found in the online
surveys of white Americans.
Thus, a professional obligation to equality, unlike a
personal commitment to the same ideal, appears to have less or no impact on automatic
racial associations, at least in judges.
This finding is consistent with the portrait of the unconscious associations of
capital defense attorneys, another group which might be expected to have strong
professional commitments to the norm of racial equality.119 Capital defense attorneys
also exhibit the same automatic preferences for whites as the general population.120
Taken together, our research on judges and the research on defense attorneys raise
serious concerns about the role that unconscious bias might play in the criminal justice
system. Jurors are drawn from randomly selected adults—a majority of whom harbor
implicit white preferences. If jurors, judges, and defense attorneys all largely harbor the
same anti-black preferences, then the system would appear to have little safeguards to
protect African-American defendants from bias.
Based on IAT scores alone, both
African-American judges and African-American jurors would seem to be preferable to
either white judges or white jurors, because African-American show less implicit bias
118
See Gordon B. Moskowitz et al, Preconsciously Controlling Stereotyping: Implicitly Activated
Egalitarian Goals Prevent the Activation of Stereotypes, 18 SOCIAL COGNITION 151, 155 (2000).
119
See Sheri Lynn Johnson & Theodore Eisenberg, Implicit Racial Attitudes of Death Penalty Lawyers, 53
DEPAUL L. REV. 1539, 1540 (2004) (“One would hope that those who represent capital defendants (or at
least African-American capital defendants) would themselves be free of racialized thinking”).
120
Id. at 1546-48.
37
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
than whites. But otherwise, an African-American criminal defendant seems vulnerable to
the influence of unconscious biases.
The rest of our results, however, call into question the meaningfulness of relying
on IAT scores alone to evaluate the racial neutrality of a legal decision maker. The
implicit racial associations that we observed did not have a consistent effect on judges’
decisions. Implicit associations influenced judges B both black judges and white judges
B when we manipulated the race of the defendant by subliminal methods. Judges with
strong white preferences on the IAT made harsher judgments of the juvenile defendants
after being exposed to the African-American subliminal prime and judges with strong
black preferences on the IAT were more lenient after exposure to the African-American
subliminal prime. In effect, unconscious channels produced unconscious bias, and in just
the way that might be expected.
The story for the explicit manipulation of race is more complicated, however.
The white judges, unlike the white adults in the Sommers and Ellsworth study, treated
African-American and white defendants equally. But the proper interpretation of this
finding is unclear, in part because of the results of the subliminal manipulation of the
defendant=s race described above, and in part because we observed the opposite
correlation between IAT score and assessment of the defendant’s guilt in the battery case.
That is, the more that white judges expressed the invidious implicit racial association, the
more favorably they treated the African-American defendant. Thus, we are not seeing a
simple picture of racially neutral decision making.
We believe that the data demonstrate that the judges attempted to compensate for
unconscious biases in their decision making. Codes of judicial conduct demand that
judges make unbiased decisions.121
Moreover, impartiality is a prominent element in
almost every widely accepted definition of the judicial role.122
The judges would
See AMERICAN BAR ASSOCIATION, MODEL CODE OF JUDICIAL CONDUCT, Canon 7(2007) (“A judge
shall perform the duties of judicial office impartially and diligently.”)
122
See, e.g., ABA Comm. on Judicial Performance Evaluation, Black Letter Guidelines for the Evaluation
of Judicial Performance with Commentary 5-2.3 (2005)(prescribing “[a]bsence of favor or disfavor toward
anyone, including but not limited to favor or disfavor based upon race, sex, religion, national origin,
disability, age, sexual orientation or socioeconomic status.”).
121
38
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
therefore have been motivated to avoid racial bias, at least when they were aware that
bias was potentially at work.
The judges in our study behaved much like the subjects in other studies who were
highly motivated to avoid bias in the assigned task.123 For example, both Glaser and
Fazio, in separate studies, found that for some subjects, the desire to control bias
suppressed, or even inverted, the relationship between implicit associations and biased
judgment.124
Similarly, the study of doctors’ diagnoses identified a reversal of the
influence of the IAT score among those doctors who were aware of the purpose of the
study B and might therefore be attempting to suppress or disguise biased reactions.125
Finally, Sommers and Ellsworth showed that some situations B situations in which the
ordinary person is alerted to the possibility that bias is being studied by an overtly racial
story B can induce an effort to control biased judgment in the entire sample.126
What made our white judges different from the subjects studied by Glaser, Fazio,
Banaji, and Ellsworth is that virtually all of the judges reported that they suspected racial
bias was being studied, despite the fact that the only cue they received was the explicit
mention of the defendant=s race. We think this report was truthful, given that they
behaved as have other white subjects who attempted to avoid the influence of implicit
bias. It is not surprising that judges are easily cued into the idea that biases are being
studied, given widespread interest in racial bias in the courts. The white judges thus
behaved like those participants in previous research who were highly motivated to control
the expression of bias.
Like the white judges, the African-American judges in our study also reported
being aware of the subject of the study, yet they showed a relationship between implicit
associations and judgment when race was explicitly manipulated. Among these judges, a
greater white preference produced a greater propensity to convict the African-American
123
124
125
126
See infra notes 43 to 51 and accompanying text.
Glaser & Knowles, supra note 43; Fazio et al., supra note 46.
Green et al., supra note 54.
Sommers & Ellsworth, supra note 51.
39
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
defendant. In other words, the African-American judges clearly reacted differently when
they were conscious that race was being manipulatedCa difference that correlated with
their score on the race IAT.
From these results, we do not conclude, however, that black judges are less
concerned about biased results than white judges. Their concern with avoiding racial
disparities in their own courtrooms might not produce the kind of compensatory
responses that we observed among white judges, because historically, the documented
inequities have disfavored their own race. African-American judges= concern with racial
disparities in the courts might therefore manifest itself in other ways. It may be that a
white judge=s worst fear is that he will appear to be partial to white defendants - and that
a black judge fears the same thing.
We do not want to overstate the import of these results. After all, this is only one
study, and it has its limitations. The results might be the product of the particular judges
who participated in our study or the materials we used or even the fact that hypothetical
scenarios were used.
Most importantly, we cannot determine whether the mental
processes of judges on the bench more closely resemble those of judges subliminally
primed by race, or those for whom race was explicitly manipulated. On the one hand,
judges see the defendant, and can be expected to have conscious awareness of his or her
race, and that fact would seem to mean that their real-world decisions are more like those
we observed when race was explicitly manipulated. On the other hand, judges in their
courtrooms have no reason to be focused on whether an observer might see evidence of
racial bias, and this might make their responses more closely resemble the results we
obtained in our study involving subliminal prompts. Thus, it is not clear how implicit
bias might influence judicial decision making in court, but our study suggests, at a
minimum, that there is some risk of this, so we turn in the next section to reforms the
criminal justice system might consider implementing.
Given our results, we cannot definitely ascribe continuing disparities in the
criminal justice system to unconscious bias.
We can draw some firm conclusions,
however. Implicit biases are widespread among judges.
40
This bias can influence their
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
judgment. Judges, however, seem to be aware of the potential for bias in themselves and
have the cognitive skills necessary to avoid its influence. Whether they engage their
abilities to avoid bias on a continual basis in their own courtrooms, however, is unclear.
B. Prescriptions
To minimize the risk that unconscious or implicit bias will lead to biased
decisions in court, the criminal justice system could take several steps. These include
exposing judges to stereotype-incongruent settings, providing testing and training,
auditing judicial decisions, and altering courtroom practices. These efforts would both
encourage judge to use the ability to avoid bias that that we have seen the judges in our
study display in our study and facilitate the reduction of unconscious biases.
1. Exposure to Stereotype-Incongruent Settings
Several scholars have suggested that society might try to reduce the presence of
unconscious biases by exposing decision makers to stereotype-incongruent settings.127
This suggestion, in fact, probably represents the dominant policy proposals of legal
scholars who write about unconscious bias. The idea that exposure to peers of different
races has also provides some of the support for affirmative action programs in higher
education.128
Our results, however, question the effectiveness of this proposal. The white
judges in the Eastern jurisdiction in our study showed a strong set of implicit biases, even
though the jurisdiction consists of roughly half white and half African-American judges.
Indeed, the level of implicit bias in this group of judges was only slightly smaller than
that of the Western jurisdiction, which had included only two African-American judges
(along with thirty-six white, five Latino, and two Asian judges). Exposure to a group of
esteemed African-American colleagues apparently was not enough to counteract the
societal influences that lead to implicit biases.
127
Kang & Banaji, supra note 11, at 1105-08; Jolls & Sunstein, supra note 4, at 988-9.
See Kang & Banaji, supra note 11, at 1112 (noting that “in Grutter v Bollinger, the Court emphasized
that student diversity was valuable because it could help ‘break down racial stereotypes’”). See also, Jerry
Kang, The Trojan Horses of Race, 118 HARV. L. REV. 1489 (2005)(arguing that public broadcasting should
be regulated so as to promote positive images of minorities).
128
41
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
2.
Testing and Training
The criminal justice system might test candidates for judicial office using the IAT
or other devices to determine whether they possess implicit biases. We do not suggest
that people who show strong preference on the IAT be barred from service as a judge, nor
do we even support using the IAT as a measure of qualification to serve on the bench.129
The direct link between IAT score and decision making is far too tenuous for such a
radical recommendation. And our data show that judges can overcome these implicit
biases. Rather, knowing a judge’s IAT score might serve two other purposes. First, it
might help prospective judges understand the extent to which they have implicit biases
and alert them to the need to correct for those biases on the job. Second, it might enable
the system to provide targeted training for judicial candidates on bias issues.130
Judicial training should not end with prospective judges, however. Training for
sitting judges is also important. Judicial training is common these days, but one problem
with it, at least as it exists at this time, is that it is seldom accompanied by any testing of
the individual judge’s susceptibility to implicit bias, or any analysis of the judge’s own
decisions, so that the judges are less likely to appreciate and internalize the risks of
implicit bias. As Timothy Wilson and his colleagues have observed, “People’s default
response is to assume that their judgments are uncontaminated;”131 surely, this is true of
judges. Moreover, because people are prone to egocentric bias, they readily assume that
they are better than average, or that matters that might induce others to make poor or
biased decisions would not affect their own decisions. Our research demonstrates that
judges are inclined to make the same sorts of favorable assumptions about their own
abilities that other people do.132 Therefore, while education regarding implicit bias as a
129
Others have made tentative suggestions that the IAT be used as screening device for certain professions.
AYRES, supra note 1, at 424 (Implicit attitude testing might itself be used as a criterion for hiring both
governmental and nongovernmental actors.”)
130
See Green et al., supra note 54, at 1237 (recommending “securely and privately administered IATs to
increase physicians’ awareness of unconscious bias”).
131
Timothy D. Wilson, David B. Centerbar & Nancy Brekke, Mental Contamination and the Debiasing
Problem, in THE PSYCHOLOGY OF JUDGMENT: HEURISTICS AND BIASES, 185-200, 190 (Thomas Gilovich,
Dale W. Griffin & Daniel Kahneman eds. 2002).
132
Chris Guthrie, Jeffrey J. Rachlinski & Andrew J. Wistrich, Inside the Judicial Mind, 86 CORNELL L.
REV. 777, 814-815 (2001).
42
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
general matter might be useful, specific training focusing on the demonstrated
vulnerabilities of the judges being trained would be more useful.133
Another problem with training is that although insight into the direction of a bias
frequently can be gained, insight into the magnitude of that bias cannot. As one group of
psychologists has explained:
Consider Ms. Green, a partner in a prestigious law firm, who is
interviewing candidates for the position of an associate in her firm.
When she interviews Mr. Jones, a young African American
attorney, she has an immediate negative impression, finding him to
be arrogant and lacking the kind of brilliance she looks for in new
associates. Ms. Green decides that her impression of Mr. Jones
was accurate and at a meeting of the partners, argues against hiring
him. She wonders, however, whether her negative evaluation was
influenced by Mr. Jones’ race.... Ms. Green may know that her
impression of Mr. Jones is unfairly negative and want to avoid by
this bias, but have no idea of the extent of the bias. Should she
change her evaluation from ‘Should not be hired’ to ‘Barely
acceptable’ or to ‘Best applicant I’ve seen in years’?134
This scenario illustrates the problem well. How is one to know if correction is
warranted, and if so, how much?135 In a circumstance like the one depicted above or like
any of the circumstances described in the materials included in our study, there is a risk
of insufficient correction; unnecessary correction; or even overcorrection, resulting in a
decision that is distorted as a result of the adjustment, but simply in the opposite
direction.136
133
See Green et al., supra note 54, at 1237.
Wilson et al., supra note 131, at 185 & 187.
135
See id,. at 191 (“Three kinds of errors have been found: insufficient correction (debiasing in the
direction of accuracy that does not go far enough), unnecessary correction (debiasing when there was no
bias to start with), and overcorrection (too much debiasing such that judgments end up bias in the opposite
direction).
136
See id. (suggesting that people’s “corrected judgments might be worse than their uncorrected ones.”);
134
43
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Finally, merely trying to suppress biases and stereotypes may result in their
“hyper accessibility”, which, paradoxically, may cause a “rebound effect” that produces
even more stereotypical judgments.137
Accordingly, “suppression is not simply an
ineffective tactic of mental control; it is counter-productive, helping assure the very state
of mind one had hoped to avoid.”138
3. Auditing
The criminal justice system could also implement an auditing program to evaluate
the decisions of individual judges to determine whether they appear to be influenced by
implicit bias. For example, judges’ decisions in highly discretionary determinations such
as bail-setting, sentencing, or child custody could be audited periodically to determine
whether they exhibit patterns indicative of implicit bias. Such proposals have been
suggested as correctives for umpires in Major League Baseball and referees in the
National Basketball Association as both groups express some evidence of racial bias in
their judgment.139
Auditing could provide a couple of benefits. First, it would obviously increase
available data regarding the extent to which bias affects judicial decision making.
Second, it could enhance the accountability of judicial decision making.140
Unfortunately, judges operate in an institutional context that provides little
accountability.
Even available forms of accountability, such as appellate review or
Anthony Page, Batson’s Blind Spot: Unconscious Stereotyping and the Peremptory Challenge, 85 B.U.L.
REV. 155, 239-240 (2005) (“One major problem for any correction strategy is determining the magnitude of
the correction required. Unfortunately, people are not very good at this determination. Some research
suggests that among those who are very motivated to avoid discrimination, over correction is a common
problem.... A second problem is that a correction strategy appears to require significant cognitive
resources....”); id. at 241-242 (“To consciously and willful regulate one’s own... evaluations [and]
decisions... requires considerable effort and is relatively slow. Moreover, it appears to require a limited
resource that is quickly used up, so conscious self-regulatory acts can only occur sparingly and for a short
time.”)(citations omitted).
137
See Wistrich et al., supra note 59, at 1262-64.
138
Page, supra note 136, at 241-42 (citations omitted)
139
Christopher A. Parsons, Johan Sulaeman, Michael C. Yates & Daniel S. Hamermesh, Strike Three:
Umpires' Demand for Discrimination (Unpublished manuscript); Joseph Price & Justin Wolfers, Racial
Discrimination Among NBA Referees (Unpublished manuscript).
140
Accountability improves performance in other contexts, so it likely would do so for judges as well. See
Jennifer Lerner & Phillip E. Tetlock, Accounting for the Effects of Accountability, 125 PSYCHOL. BULL.,
255 (1999).
44
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
retention elections, primarily focus on a judge’s performance in a particular case, not on
the systematic study of long-term patterns within a judge’s performance that might reveal
implicit bias.
4. Altering Courtroom Practices
In addition to providing training or implementing auditing programs, the criminal
justice system could also alter practices in the courtroom to minimize the untoward
impact of unconscious bias.
For example, in court proceedings, the system could
disguise litigants’ race from the judge. Research suggests that when exposure to a
stimulus produces a bias or a cognitive error, subsequent remedial measures are a poor
second best to avoiding exposure to the stimulus altogether.141 Accordingly, some have
suggested that efforts might be made to avoid revealing the race or gender of one or both
litigants to the fact finder.142
This suggestion may not be as far-fetched as it seems. Similar approaches have
worked well in other contexts. The use of audition screens in the 1960s and 1970s, for
example, helped to increase the number of women and minorities employed by
symphony orchestras.143
On the other hand, attempting to prevent judges from learning
the race or gender of a litigant would introduce a degree of artificiality into legal
proceedings that many would find intolerable. Often it would be impossible. Witnesses
might have to refrain from referring to litigants by their first name (which correlate with
race), exhibits might have to be redacted or altered, and so on.
See Wilson et al., supra note 131, at 192 (“Exposure control, to the extent that it is feasible, is the most
effective defense against contamination.”); id., at 195 (“The best way to avoid biased judgments is
exposure control, whereby we avoid stimuli that might influence our responses in unwanted ways.”);
Wistrich et al., supra note 59, at 1325-27.
142
See, e.g., Note, Trumping The Race Card: Permitting Criminal Defendants to Remain Anonymous and
Absent from Trial to Eliminate Jury Bias, 18 GEO. J. LEGAL ETHICS 1151, 1156 (2005) (“By maintaining
the anonymity of a criminal defendant and making all efforts to prevent the jury from learning the race of
the defendant (in most situations, at least where the defendant’s race is not a relevant issue), the opportunity
for racial factors to obfuscate the issues of guilt or innocence would be eliminated.”); see also Jaime N.
Morris, Note, The Anonymous Accused: Protecting Defendants’ Rights in High Profile Criminal Cases, 44
B.C. L. REV. 901, 945 (2003) (arguing that shielding the identity of high-profile or celebrity criminal
defendants would protect them from jurors’ prejudicial preconceived notions).
143
See Robert Post, Prejudicial Appearances: The Logic of American Antidiscrimination Law, 88 CAL. L.
REV. 1, 14-15 (2000); American Orchestras: All Ears, THE ECONOMIST, Nov. 30, 1996, at 8.
141
45
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Another possibility would be to expand the use of three-judge courts. Research
reveals that improving the diversity of appellate court panels can affect outcomes. For
example, one study found that “adding a female judge to the panel more than doubled the
probability that a male judge ruled for the plaintiff in sexual harassment cases . . . and
nearly tripled this probability in sex discrimination cases.”144 In trial courts, judges
typically decide such issues alone, so this mechanism requires major structural changes.
Although convening a three-judge trial court was once required by statute when the
constitutionality of a state’s statute was at issue, three-judge trial courses are virtually
nonexistent today. The inefficiency of having three judges decide cases that one judge
might be able to decide nearly as well led to their demise and this measure might simply
be too costly to resurrect.
Nevertheless, expanding the use of three-judge courts,
particularly in cases in which race is likely to be a salient issue, might reduce the impact
of implicit biases in the cases where they are especially likely to be triggered.
Yet another possibility would be to increase the depth of appellate scrutiny, such
as by employing de novo review rather than clear error review, in cases in which
particular trial court findings of fact might be tainted by implicit bias. For example, there
is some evidence that male judges may be less hospitable to sex discrimination cases than
they ought to be.145 If true, less deferential appellate review by a diverse panel might
offer a partial solution.
None of these is a panacea, but each is worth contemplating, particularly if further
research suggests that implicit biases can, in fact, lead to injustice in the criminal justice
system.
144
Jennifer L. Peresie, Note, Female Judges Matter: Gender and Collegial Decision Making in the Federal
Appellate Courts, 114 YALE L.J. 1759, 1778 (2005).
145
See id., at 1778.
46
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
V. Conclusion
Our study contains both bad news and good news about implicit biases among
judges. As expected, we found that the judges, like the rest of us, possess implicit biases
and that these biases have the potential to influence judgment in the criminal justice
system, at least in those circumstances where judges are not guarding against their
effects. On the other hand, we found that the judges managed, for the most part, to avoid
the influence of unconscious biases when race was made explicit. At a minimum, the
presence of implicit biases among judges – even if their impact on actual cases is
uncertain — should sound a cautionary note for those involved in the criminal justice
system.
47
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Appendix A: Materials
Shoplifter Case
You are presiding over a case involving criminal charges against a juvenile, William T. William
is a 13-year old who was arrested for shoplifting in a large, upscale toy store in _____. He has no
prior record. You are trying to get a sense of the case and the only facts available to you follow:
According to a store clerk, on Saturday, April 2 at about two o’clock in the afternoon, the clerk
observed William putting video games under his shirt. The clerk rang for a security guard, but
before the guard arrived, the boy started to leave the store. When the clerk grabbed William, the
boy dropped the toys and kicked him in an attempt to escape. A uniformed security guard arrived
as the clerk let go of William, and when the guard told the boy to stop, he did.
According to the security guard, when he arrived he observed five items on the floor in front of
William. The prices of those items together added up to $90. He said that William told him that
he was shopping, and showed him $10 he had brought along with which to make purchases.
William claimed that he had used his shirt as a sort of pouch to hold the items he was looking at.
William also told the guard he was startled when grabbed by someone from behind, and then
tripped, but that he did not kick anyone.
1. In your opinion, without regard to the options actually available in this kind of situation, what
would be the most appropriate disposition of this case?
_____ 1) dismiss it with an oral warning
_____ 2) adjourn the case in contemplation of dismissal (assuming William gets
in no further trouble)
_____ 3) Put William on probation for six months or less
_____ 4) Put William on probation for more than six months
_____ 5) Commit William to a juvenile detention facility for six months or less
_____ 6) Commit William to a juvenile detention facility for more than six months
_____ 7) Transfer William to adult court
48
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
2. In your opinion, on a scale of one seven, how likely is it that William will later commit a
crime similar to the one with which he is charged?
Very
Not at all
Likely
Likely
1
2
3
4
5
6
7
3. In your opinion, on a scale of one to seven, how likely is it that William will commit
more serious crimes in the future?
Very
Not at all
Likely
Likely
1
2
3
4
49
5
6
7
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Robbery Case
You are presiding over a case involving criminal charges against a juvenile, Michael S,
who was arrested for armed robbery of a gas station when he was two days shy of his seventeenth
birthday. He has one prior arrest for a fight in the school lunchroom the previous year. You are
trying to get a sense of the case and the only facts available to you follow:
According to the gas station clerk, on Friday, March 17, at about seven in the evening,
she heard a male voice say, A Don=t look at me, but give me the money.@
She kept her eyes
down, and as she opened the cash register, the man said, AI could shoot you, don=t think I
won=t.@ She handed him the drawer=s contents ($267. 60) and saw him run out the door with a
gun. After he jumped into the passenger side of a car and it left, she called the police.
According to the responding officer, the clerk could not identify the robber, but a
customer said he thought he recognized Michael, and gave the officer Michael=s name and
address. Michael=s mother was home, and at nine-forty five, Michael walked in the door ,was
given Miranda warnings, and waived his rights. He first stated that he had just been hanging
around with friends, not doing anything special. After the officer asked who the friends were,
Michael admitted that he had walked into the gas station with a gun. He told the officer that he
said to the clerk, A Give me the money, please, I don=t want to hurt you.@ Michael insisted that
the gun was not loaded, and that he no longer had it. He said that the money was gone, that he
was sorry, and he would pay it back. When asked why he did it, Michael said that his friends had
dared him, but he would not reveal who those friends were, or to whom the gun belonged.
1. In your opinion, without regard to the options actually available in this kind of situation, what
would be the most appropriate disposition of this case?
_____ 1) dismiss it with an oral warning
_____ 2) adjourn the case in contemplation of dismissal (assuming Michael gets
in no further trouble)
_____ 3) Put Michael on probation for six months or less
_____ 4) Put Michael on probation for more than six months
_____ 5) Commit Michael to a juvenile detention facility for six months or less
_____ 6) Commit Michael to a juvenile detention facility for more than six months
_____ 7) Transfer Michael to adult court
50
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
2. In your opinion, on a scale of one to seven, how likely is it that Michael will later commit a
crime similar to the one with which he is charged?
Very
Not at all
Likely
Likely
1
2
3
4
5
6
7
3. In your opinion, on a scale of one to seven, how likely is it that Michael will commit more
serious crimes in the future?
Very
Not at all
Likely
Likely
1
2
3
4
51
5
6
7
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Expert Witness
You are presiding over a product liability suit by Nathan Reuben, an employee of a local
dry-cleaning shop, against Pure Chemicals Inc., a chemical manufacturing company. The case
will be tried by jury.
Reuben contends he suffers from significant neurological impairment, including
cognitive difficulties, memory lapses, and so forth, caused by exposure to the chemicals that Pure
Chemicals, Inc. produced and sold to the dry-cleaning shop where he works. Pure Chemicals Inc.
disputes this and plans to offer expert medical testimony that Reuben has suffered no impairment,
as measured by any standard technique.
Reuben wants to offer expert testimony from Patricia/Patrick Knoll.
Knoll has a
bachelor’s degree in Chemistry, a Master’s degree in Statistics and a Ph.D. in Neuroscience from
the Hunter Hills University in Pitford, Louisiana. S/he is a general science instructor at a local
community college. S/he has written one only scholarly article, which describes a novel means of
mathematically combining the results of PET scans and MRI measures of neurological activity.
Her/His claim is that her/his procedure provides a more reliable means of detecting cognitive
deficits caused by chemical exposure. S/he will testify that s/he applied her procedure to the
results of PET scans and MRI measures of Reuben’s brain and found cognitive deficits that, in
her/his professional opinion, are likely to have been caused by the dry-cleaning chemicals
produced and sold by Pure Chemicals Inc.
The defendant has moved to exclude this evidence on the ground that Knoll does not
“have adequate knowledge, or experience” in the relevant field “to ensure that the opinion will
assist the trier of fact.” [Note: Cites from the jurisdiction omitted and quotes have been slightly
altered in this report to make it impossible to identify the source.] They label her/his procedure
“untested”, “novel”, and “junk science.”
How would you rule on the defendant=s motion to exclude testimony from Knoll?
______ Grant the motion and exclude the testimony
______ Deny the motion and admit the testimony
52
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
How confident are you that your judgment is correct?
Very
Not at all
Confident
Confident
1
2
3
4
53
5
6
7
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Battery Case
Defendant:
André Barkley, 6=0@, 175 lbs., African-American male, 18 years-old, student
Alleged Victim: Matthew Clinton, 6=2@, 185 lbs., Caucasian male, 16 years-old, student
Charge:
One Count of Battery with Serious Bodily Injury
Prosecution
The prosecution claims that André Barkley is guilty of battery with serious bodily injury.
Barkley was the starting point guard on the high school basketball team, but the team had been
struggling, and the coach decided to bench him in favor of a younger, less experienced player
named Matthew Clinton. Before the first game after the lineup change, Barkley approached
Clinton in the locker room and began yelling at him. Witnesses explain that the frustrated
defendant told Clinton that Ayou aren=t half the player I am, you must be kissing Coach=s ass
pretty hard to be starting.@
When other teammates stepped between the two players, Barkley told them to get out of
the way. When two other players then grabbed Barkley and tried to restrain him, the defendant
threw them off, pushed Clinton into a row of lockers, and ran out of the room, according to
prosecution witnesses. As a result of this fall, 2 of Clinton=s teeth were chipped and he was
knocked unconscious. The prosecution claims that Barkley has shown no remorse for his crime,
and has even expressed to friends that Clinton Aonly got what he had coming.@
Defense
The defense claims that Barkley was merely acting in self-defense, and that Clinton=s
injuries were accidental. According to an assistant coach, Barkley did not get along with many
people on the team and had been the subject of obscene remarks and unfair criticism from many
of his teammates throughout the season. Barkley claims that he was afraid for his own safety
during the altercation in the locker room and Adefinitely felt ganged up on.@
Barkley admits he Amight have been aggressive towards Matthew and started the whole
thing,@ but says that he was just frustrated and the argument was Anothing that should have
54
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
started a big locker room fight or anything.@ Barkley claims that when several other players
grabbed him from behind for no reason, he tried to break free and must have accidentally
knocked into Clinton in the attempt to get out of the locker room. He explained that the reason he
never apologized to Clinton in the hospital was that he Adidn=t think he=d want to see me,@ but
Barkley did say he Awas truly, truly sorry@ that Clinton had been injured.
Based on the available evidence, if this were a bench trial, would you convict the defendant?
Yes
No
How confident are you that your judgment is correct?
Very
Not at all
Likely
Likely
1
2
3
4
5
55
6
7
8
9
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Appendix B: IAT Procedure
We used seven rounds of trials to produce the IAT score. Rounds 1, 2, 3, 5, and 6
are essentially practice rounds designed to minimize order effects and variation
associated with unfamiliarity with the task. The study begins with one round in which
the participants only sort black and white faces. In this round the word “White” appeared
in the upper left and the word “Black” appeared in the upper right of the screen. In each
trial, one of ten faces, five white and five African-American, appeared in the middle of
the screen.146 The faces appeared at random, although an equal number of white and
black faces appeared in the 16 trials.147
The instructions before each round informed the judges as to what they would be
sorting in the upcoming round. For example, in the first round, the instructions indicated
that the judge should press the “E” key (labeled with a red dot) if a white face appeared
and the “I” key (also labeled with a red dot) if an African-American face appeared. The
materials also state that if the judge pressed the correct key, the next face would appear;
if the judge pressed the wrong key, a red “X” would appear. These instructions were
similar in all seven rounds of the IAT.148
The remaining six rounds were similar to the first, although they varied the
stimuli and categories. In the second round, instead of the black and white faces, the
146
The faces were taken from the Project Implicit website and have been rated as highly identifiable as
black or white. (www.projectimplicit.org) They include only the center of the face, with ears, hair, and
anything below the chin cropped out. None of the faces have facial hair, eyeglasses, or distinguishing
features.
147
In this respect we varied from the procedures recommended by Greenwald et al. (2003), by reducing the
practice rounds from the 20 they suggest to 16. We did this in the interests of saving time. We did retain
the 40 trials in the critical rounds. We had more time available in the Western jurisdiction, and increased
the length of rounds 3 and 6 to 20.
148
The exact instructions were as follows:
In the first round, the two CATEGORIES that you are to distinguish are:
BLACK vs. WHITE faces.
Press the 'e' key if the TARGET is a WHITE face.
Press the 'I' key if the TARGET is a BLACK face.
Remember that an "X" will appear when you make an error. Whenever the "X" appears, correct
the mistake by pressing the other key.
Please respond AS RAPIDLY AS POSSIBLE, but don't respond so fast that you make many
errors. (Occasional errors are okay.)
Press the space Bar when you are ready to begin
56
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
computer presented good and bad words. These consisted of seven words with positive
associations (Joy, Love, Peace, Wonderful, Pleasure, Friend, Laughter, Happy) and seven
words with negative associations (Agony, Terrible, Horrible, Nasty, Evil, War, Awful,
Failure).
Like the faces, these words were taken from previous work on the IAT.
Throughout the trials in the second round, the word “Good” remained in the upper-left of
the computer screen and the word “Bad” remained in the upper-right of the computer
screen. The Judges were instructed in a similar fashion to round one, to press the “E” key
when a good word appeared in the center of the screen and to press the “I” key when a
bad word appeared in the center of the screen.
The third round combined the tasks in the first two rounds. The words “White or
Good ” appeared in the upper-left of the computer screen and the words “Black or Bad”
appeared in the upper-right of the computer screen.
Thus, the task presented both
categories in the same spatial location as they had been in the first two rounds. The
instructions indicated to the judge that either a white or black face or a good or bad word
would appear in the center of the computer screen. The instructions continued that the
judges should press the “E” key if either a white face or a good word appeared and the “i”
key if either a black face or a bad word appeared. Although the computer selected
randomly from the faces and concept words, the computer presented an equal number of
names and faces of both types. We presented the judges with 16 trials of this task
Round four was identical to round 3 in every respect except that the computer
presented 40 trials, rather than 16.
Round five prepared the judges for the reverse association. To create the reversal,
the spatial locations of the good and bad words were reversed. The word “Bad” was
moved to the left and the word “Good” was moved to the left. The fifth round was thus
identical to second round in that the computer presented only the good and bad words,
but that the computer presented the words in their new locations. The instructions were
also identical to those of round two except that they identified the new locations and
corresponding response keys for the words.
57
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
The penultimate round paired the good and bad words in their new locations with
the black and white labels in their original location. Thus, the words “White or Bad”
appeared in the upper left and the words “Black or Good” appeared in the upper right.
The instructions resembled those for rounds 3 and 4. They indicated, however, that
judges should press the “E” key if a white face or bad word appeared and to press the “I”
key if a black face or good word appeared. Round six, like the other practice rounds,
consisted of 16 trials.
Round seven was identical to round six in every respect except that the computer
presented 40 trials, rather than 16.
The computer recorded the reaction times between the presentation of the stimuli and the
time of the correct response for all judges in all rounds. The computer also recorded
which stimuli it presented and whether an error occurred.
58
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Appendix C: IAT Scoring
As noted in the text, we used two different scoring methods: a mean difference
score and a standardized difference score. Both are described below.
1. Mean-Difference IAT Score Calculation
To calculate the mean-difference IAT score, we largely followed the procedures
outlined in Nosek et al’s report of IAT scores from tens of thousands of people collected
through the Internet.149 We also wanted to compare our results with the more detailed,
contemporary Internet data collected and reported on the “Project Implicit” website,
which appears to use the same scoring method. Because the data in these studies come
from voluntary participants who access the site on the Internet, the authors have adopted
a number of techniques for excluding data from participants who may have wandered off
during the study or are otherwise not fully engaged with the tasks. While such techniques
are less appropriate for our participants, who were engaged in person, we followed their
scoring methods to facilitate a comparison.
These authors of the Internet study first adjusted raw latency scores that seemed
much slower or faster than participants who are fully engaged with the task.
The
researchers treat any latency larger than 3,000 milliseconds (“ms”) as 3,000 ms, and any
latency shorter than 300 ms as 300 ms.150 The researchers also eliminated the first two
trials in all rounds from considerations, having found that these rounds often displayed an
erratic pattern of long latencies—presumably because participants commonly begin the
task, and then pause to settled in. These researchers also exclude participants who failed
to perform to certain criteria. They excluded participants who exhibited overall average
latencies in the two critical rounds greater than 1800 ms, or who displayed average
latencies in either of the two critical rounds (4 or 7) greater than 1500 ms. They also
excluded participants who produced any critical round in which more than 25% of the
latencies were less than 300ms. Finally, they excluded participants who made more than
10 errors in any critical round. These researchers report that these criteria resulted in the
149
150
Nosek et al., supra note 19.
Id., at 104.
59
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
exclusion of 13% of their subjects.
After these adjustments and exclusions, these
researchers calculated the mean difference between the critical stereotype-congruent
round (either round 4 or 7) and the stereotype-incongruent rounds (either round 4 or 7).
We followed these procedures to calculate the mean IAT score for the judges in
our study. We capped latencies greater than 3000 ms as 3000 ms, and raised latencies
lower than 300 ms to 300 ms.151 We also discarded the first two rounds from the
analysis. We excluded the results of the race IAT from six judges (or 4.5%) who
produced either mean latencies greater than 1800ms in one of the two critical rounds of
the race IAT or a mean across both rounds greater than 1500ms.152
Similarly, we
excluded the results of the sex IAT from ten judges (or 7.5%) who violated one or both of
these criteria.153 Nosek et al. reported that they eliminated 2% of their participants for
being too slow, whereas we eliminated more. At the same time, none of the judges in our
studies produced more than a 25% error rate in either of the critical rounds in either IAT.
By contrast, Nosek et al. eliminated roughly 10% of their participants for having high
error rates. The judges were thus slower and more accurate than Nosek et al. subjects,
and overall, the application of Nosek et al’s criteria eliminated fewer judges than Nosek
et al’s results would have predicted.
Unlike the Nosek et al. study, we did not randomize the order in which we
presented the IAT. That is, roughly half of the judges in the Internet sample receive the
151
None of the judges provided latencies that were less than 300ms in either of the two critical rounds
measuring the race IAT; two of the judges provided responses that were faster than 300ms in the genderIAT (one round each). Many more of the judges produced latencies that exceeded 3,000ms. On the raceIAT, 58 judges (or 50.4%) produced at least 1 latency greater than 3,000ms in the stereotype-congruent
round (round 4). Specifically, in the stereotype-congruent round: 33 judges produced 1 long latency; 20
produced 2; 3 produced 3; and 2 produced 4. In the stereotype-incongruent round on the race-IAT (round
7), 68 judges (or 59.1%) produced at least 1 latency greater than 3,000ms. Specifically, in the stereotypeincongruent round: 33 judges produced 1 long latency; 12 produced 2; 10 produced 3; 4 produced 4; 2
produced 5; 4 produced 6; and 3 produced 7. On the gender-IAT, 57 judges (or 49.6%) produced at least 1
latency greater than 3,000ms in the stereotype-congruent round (round 7). Specifically, in the stereotypecongruent round: 36 judges produced 1 long latency; 7 produced 2; 9 produced 3; 3 produced 4; 1 produced
5; and 1 produced 8. In the stereotype-incongruent round on the gender-IAT (round 4), 56 judges (or
48.7%) produced at least 1 latency greater than 3,000ms. Specifically, in the stereotype-incongruent round:
27 judges produced 1 long latency; 15 produced 2; 6produced 3; 3 produced 4; 2 produced 5; 1 produced
6; and 1 produced 7. Note that because some of these long latencies fell into the first two rounds, they are
not included in the analysis.
152
One of the judges violated both criteria. We calculated both means after excluding the first two
rounds.
153
Four judges violated both criteria.
60
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
stereotype-congruent round first, while half receive the stereotype-incongruent round
first. The 7-round IAT is designed to reduce order effects substantially, but nevertheless,
they remain. Greenwald et al, report that the IAT scores can correlate weakly with the
order in which the materials are presented.154
Randomizing the order would have
produced a cleaner measure of the IAT effect across all judges, but would have reduced
the correlation between the IAT score and behavior.155 Hence, all of our judges received
the materials in the same order.
On the race-IAT, judges receive the stereotype-
congruent pairing first (white/good and black/bad) and on the gender-IAT, judges receive
the stereotype-incongruent pairing first (male/humanities and female/science).
Our
produced would have tended to increase the IAT score on the race-IAT, as compared to
the Nosek et al sample, and decrease the IAT score on the gender-IAT.
By using these procedures, we scored judges in exactly the same method as Nosek
et al. in the data that they harvested from the Internet. Because laboratory data is
obviously different in some respects, we only treated the data this way for purposes of
comparison with the Internet samples, and not for assessing the correlation between the
IAT scores and the decisions that judges made. For the correlations, we calculated a
standardized score.
2. Standardized IAT Score Calculation
To calculate the standardized IAT score, we followed the procedures
recommended by Greenwald and his colleagues.156 These researchers designed their
methods precisely to improve reliability and predictive power of their measures. We use
154
Greenwald et al., supra note 35, at 210 (Table 2) report the effect of order with a correlation
coefficient, rather than a mean or percent difference. They report that the correlation varies with the IAT,
noting that the gender IAT that we used here produces a higher correlation between order and IAT score
than other IATs. Greenwald et al report correlations as high as .29 (depending upon the scoring method),
which would mean that order can account for up to 10% of the IAT score. By contrast, the race IAT that
we used produces small correlations with order, ranging from .002 to .054; thus account for, at most, one
quarter of one percent of the IAT score. The order effects seem to vary with context, and hence we cannot
be certain of the extent of the influence of order on our materials.
155
Had we randomized the order, each judge’s IAT score have varied with the order to some extent. This
would have introduced some variation to the IAT score that would inherently reduce the correlation we
observed across all judges. Our measure of the IAT score across all judges would have been more reliable
had we randomized, but the IAT score for the individual judges would have been less consistent, thereby
interfering with the correlation.
156
Greenwald et al., supra note 35.
61
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
the methods that produced the highest correlations between implicit measures and
behavioral measures. It differs from the scoring method used to calculate the mean
differences. As noted above, we used their methodology to collect the IAT scores.157
Following their scoring procedures, we removed single trials with latencies greater than
10,000 ms (that is, ten seconds) from the analysis. We otherwise left low and high values
in the analysis without adjustment. We made no correction for errors, because our IAT
collection methods required the judges to provide the correct response before proceeding
and hence the latency includes the delay that would result from an incorrect answer.
Error rates were also low, as noted above. Following Greenwald et al’s scoring method,
we used all of the trials, rather than drop the first two in the round.
We departed from the method Greenwald et al endorse, however, in one respect.
These researchers suggest using the two paired practice rounds (rounds 3 and 6) in the
analysis. They reported that using this data produced slightly higher correlations between
the IAT scores and explicit choices. We found, however, that latencies in the practice
rounds were highly erratic. A high percentage of the trials eliminated for being greater
than 10,000 were in the trial rounds were in the trial rounds.158
Even with these
observations removed, the average standard deviation in the two practice rounds on the
race IAT was over one second (1,064ms), as compared to 596ms in the trial rounds. This
suggested to us that we ought not to use the practice rounds in the analysis. The practice
rounds gender IAT were more stable. The standard deviation from the practice rounds
(724ms) was much closer to that of the trial rounds (560ms). Even though the practice
rounds in the gender IAT seemed more stable, for consistency, we dropped these as well.
Our measure of the IAT effect for purposes of correlating the IAT scores with judges’
decisions was therefore the average difference between the stereotype-congruent round
and the stereotype incongruent round divided by the standard deviation of latencies in
both rounds combined. Following Greenwald et al, call the measure d’.
157
Note differences in number of trials in practice rounds.
In the race IAT, 29 out of the 33 instances in which judges produced latency scores of greater than
10,000 ms on a trial (or 87.9%) occurred during the practice rounds. In the sex IAT, the two instances in
which judges exhibited trials that exceeded 10,000ms occurred in the target round.
158
62
DOES UNCONSCIOUS BIAS AFFECT TRIAL JUDGES?
Because the latencies that we observed seemed slower than that which has been
observed in the Internet study, we assessed the correlation between our two IAT measures
and the mean latency. The correlation coefficients between the mean differences and the
overall latency were .305 on the race IAT and .361 on the gender IAT. These correlations
are high enough to indicate that our judges will have higher IAT scores than other
populations simply because they were somewhat slower.159 The standardized IAT measure
using only the trial rounds, however, produced correlations of only .046 and .002 with the
overall mean latencies for the race and sex IATs, respectively. Hence, the d’ measure
provided a much more reliable measure of the IAT effect than the mean difference.
159
Note that these correlations used all judges, with no exclusions for speed, did not bound the data
between 300 and 3,000, and did not excluded the first two rounds, as we did for calculating the mean
differences.
63