Blinking on the Bench:
How Judges Make Decisions
Jeffrey J. Rachlinski
Cornell Law School
Business Litigation Section,
Dallas Bar Association
April 12, 2011
My Thesis
• Judges rely on simple rules of thumb that produce
intuitions about how to decide cases
• These intuitions can be accurate, but often lead to
error
• Accurate judging requires that judges engage in a
deliberative assessment of the cases before them
Compare: Psychology of
Judgment and Choice
• System 1 -- intuitive, associative, affective,
rapid judgment
– Legal Realism?
• System 2 -- deliberative, rule-based,
calculating, mathematical, deductive, slow
judgment
– Legal Formalism?
Judging: Generally and in the
Courtroom
• In ordinary life, people must use both
System 1 and System 2
– But System 1 is faster, and a bit less conscious
– Knowing when to suppress intuition is essential
to sound judgment
• Why would judgments made in the
courtroom be any different?
Our Research Program
• Over 3,000 trial judges & 400 lawyers
• Participating in judicial education conferences
• Hypothetical questions
Are Judges Different?
The Cognitive-Reflection Test
• Frederick & Kahnman (2002)
• 3 questions
• Participants asked to answer them “as best
as you can”
CRT: Question #1
• A bat and a ball together cost $1.10. The
bat costs $1.00 more than the ball. How
much does the ball cost?
Intuition: 10 ¢
Answer: 5 ¢
CRT: Question #2
• If it takes 5 machines 5 minutes to make 5
widgets, how long would it take 100
machines to make 100 widgets?
Intuition: 100 minutes
Answer: 5 minutes
CRT: Question #3
• In a lake, there is a patch of lily pads.
Every day the patch doubles in size. If it
takes 48 days for the patch to cover the
entire lake, how long would it take for the
patch to cover half of the lake?
Intuition: 24 days
Answer: 47 days
CRT: Three Basic Findings
1. Even though the questions are not difficult,
most people get most of them wrong.
2. The wrong answer most commonly chosen
tends to be the intuitive one
3. Those who get a problem wrong tend to
think it an easier problem than those who
get it right
CRT & Judges: Would they
Perform Differently?
• Maybe judges are, by nature, System 2
people
– Such as engineering students
• 300 trial judges in Florida
CRT Results in Judges
1. Judges got most of the questions wrong:
-Average score 1.23 out of 3.00
2. The most common wrong answers were
the intuitive ones
-chosen by 97%, 60% & 58% of the judges
3. Judges who chose the intuitive answer
thought the problem was easier
-e.g., 90% v 66% on question #1.
Judges Versus Others on CRT
MIT students
Carnegie Mellon students
Harvard students
Florida trial judges
Web-based participants
---- State appellate judges
M----- State students
2.18
1.51
1.43
1.23
1.10
0.95
0.79
The (Unfortunate) Headline:
“Judges Flunk Math Test”
-American Bar Association Journal
Judges’ Responses
• “SO WHO CARES about widgets?? What does
this have to do with “who gets custody of a child”
• “These questions have nothing to do with making
decisions.”
Can Intuition Lead Judges
Astray in Legal Settings?
1.
2.
3.
4.
Exemplar Cuing
Anchoring
Consistency seeking
Confirmation Bias
1. Exemplar Cuing
Context highlights different aspects of a fact
pattern
“Jellybean Phenomenon”
• Suppose I’ll give you $20 if you draw a red
jellybean from an urn filled with red and
white jellybeans
Choice of Two Urns
Urn #1: 1 red 9 white
Urn #2: 10 red 90 white
Civil Commitment & Judges
-Monahan & Silver 2003
• Standard: danger to self or others
• Description of a mentally disturbed patient
• What threshold risk of violence would
justify a civil commitment?
– Using two different (but identical) scales:
• % chance of a violent act
• n/100 people like this commit a violent act
Civil Commitment: Format Variations
Risk of violence (in %) :
1% 8%
26%
56%
76%
Risk of violence (n/100):
1 8
26
56
76
Sentencing Problem
• 136 Trial Judges at National Conference in US
• Criminal sentencing decision
• Defendant stabbed victim to death after victim
taunted him about fiance
• Defendant is guilty of voluntary manslaughter
• Without regard to your jurisdiction, what is the
appropriate sentence?
Sentencing Format Variation
______/Years
______/Months
Average Sentencing by Format
Years
10
9
8
7
6
5
4
3
2
1
0
120
108
96
84
72
60 Months
48
36
24
12
0
2. Anchoring
The tendency to base numeric estimates on
numeric reference points, even when the
reference points are random
Anchoring: An Example
• What is the average price of a textbook in
the campus bookstore?
• Before you answer, is it more or less than
$103,471
Anchoring in Judges
•
•
•
•
Civil rights violation
Defendant is a public-sector employee
Plaintiff is a secretary
Supervisor calls her racial epithets and ridicules
her ancestry in front of co-workers and her
daughter
• Plaintiff finds other position, but City human
rights commission brings case on her behalf
• Only damages are for “mental anguish”
The Anchor
• No Anchor—Plaintiff asserts that she recently saw
a case similar to hers on a “court television show
where the plaintiff received a compensatory
damage award for mental anguish.”
• Anchor—Plaintiff asserts that she recently saw a
case similar to hers on a “court television show
where the plaintiff received a compensatory
damage award of $415,300 for mental anguish.”
Irrelevant Anchor: Results
(Median Award, in $ thousand)
70
60
50
40
30
20
10
0
No Anchor
Anchor
Irrelevant Anchor #2:
Personal Injury Suit (large)
(Median Award, in $ thousand)
1,000
900
800
700
600
500
400
0
No Anchor
Anchor ($75k)
Irrelevant Anchor #3:
Personal Injury Suit (small)
(Median Award, in $ thousand)
100
90
80
70
60
50
40
0
No Anchor
Anchor ($332k)
Irrelevant Anchor #4:
Sentencing Order
• Military trial judges sentencing 2 defendants
– Threatening with an unloaded weapon (3 years)
– Voluntary manslaughter (15 years)
• Order of sentencing varied
– Half sentenced “threat” then “manslaughter”
– Half sentenced “manslaughter” then “threat”
• 75% of appellate judges stated order has no effect
Sentencing (Threat): Results
(average sentence in years)
Sentencing (Manslaughter): Results
(average sentence in years)
3. Consistency Seeking
• The tendency to see the facts and law in
ways that are consistent (even when they
need not be consistent)
Immigration Case
• Defendant, a Peruvian citizen, pled guilty to using a false
identification document, a misdemeanor
Forged U.S. entry visa was pasted into genuine
Peruvian passport
• Two different reasons for attempting to enter United States
illegally, either
Father: find a job to pay for daughter’s liver
transplant
Killer: track down a rogue member of a cartel who
had stolen drug proceeds
Immigration Case
• Legal issue of first impression: does
pasting a forged U.S. entry visa into a
genuine foreign passport constitute
“forgery of an identification card”
• If so, 6 months in prison (deportation
follows either way)
Consistency Seeking: Results
% finding against defendant?
Consistency Seeking in New Judges
% finding against defendant?
4. Confirmation Bias
• Tendency to seek information that is
consistent with a hypothesis—even when
that information is logically irrelevant to
that hypothesis
Card-Selection Problem
• “If there is a vowel on one side of the card, then there is an even
number on the other side.”
E
X
P 4
7
X
•Which one(s) must you turn over to test the hypothesis?
Logical Problem
• “If there is a vowel on one side of the card, then there is an even
number on the other side.”
• 4 cards:
–
–
–
–
1: the letter “E” on one side and the number ___ on the other
2: the letter “P” on one side and the number ___ on the other
3: the number “4” on one side and the letter ___ on the other
4: the number “7” on one side and the letter ___ on the other
• Which one(s) must you turn over?
Gender Discrimination Problem
• “male supervisors only promote male employees”
• 4 people:
–
–
–
–
1: a male supervisor who recently promoted a ___ employee
2: a female supervisor who recently promoted a ___ employee
3: a male employee who was recently promoted by a ___ supervisor
4: a female employee who was recently promoted by a ___ supervisor
• Which one(s) must you identify?
Logical/Discrimination Problem:
Previous Results
(% Correct)
• Logical Problem:
9% correct
• Discrimination Problem:
14% correct
Addendum:
What About Lawyers?
• “Decision makers often pursue noninstrumental information—information that
appears relevant but, if simply available,
would have no impact on choice. Once they
pursue such information, people then use it
to make their decision.”
– Bastardi & Shafir, 1998
Useless Information: Materials
•
•
•
•
•
•
Plaintiff is a worker in auto-assembly plant
Injured by defendant’s robotic welding unit
Serious facial scaring
Defendant offers to settle for $400,000
Plaintiff feels it is low, but reluctant to take risks
Government safety report on cause of the
accident
The “Useless” Information
• Simple version
– Government report blames the manufacturer,
rather than the worker
– Settle for $400,000?
• 2-Stage version
– Report is not out yet—should you wait?
– If wait, told that it blames manufacturer—
should you settle?
Useless Information: Results
• Simple version:
62% settle
– The report is irrelevant to this group
• 2-Stage version:
55% wait for report
• Once report is out: 23% settle
Conclusions
• Intuitive thinking can lead judges astray,
even in legal contexts in which they are
expert
• Mechanisms to engage the slower,
deliberative system are essential to quality
judicial decision making
“When all is said and done, we
must face the fact that judges are
human.”
-Jerome Frank, 1949