The Use and Abuse of Empirical Evidence
in Support of Normative Arguments on Judicial Selection
Charles Gardner Geyh1
I am not a social scientist by training and have not produced quantitative
empirical research of my own. My scholarship on judicial selection is situated at the
intersection of law and politics, and it is from that vantage point that I offer a few
modest observations on what empirical research does—and does not—tell us about
how judges should be selected. I have four points to make, and while those points
are not unrelated to each other, neither do they converge to produce a synthetic
whole that exceeds the sum of its parts. The first three points concern ways in
which empirical research informs the normative debate, and the fourth concerns
ways in which it does not.
1. Exposing the Errors of Stridency
"The fundamental cause of trouble in the world is that the stupid are
cocksure while the intelligent are full of doubt."
- Bertrand Russell
Scholars love to attack accepted wisdom. It gets us noticed and it earns us
tenure. But there are reasons why accepted wisdom is accepted as wisdom, not the
least of which being that it has withstood the test of time. That hardly makes
accepted wisdom right, but it entitles accepted wisdom to occupy the default
position until disproven.
When it comes to judicial selection, there is not now and never has been a
consensus on how best to select judges in the United States. The debate has raged
since the constitutional convention. Durable uncertainty is its own form of accepted
wisdom, nicely captured in a widely quoted line attributed to a Texas Chief Justice
(whose name has been lost in the mists of time): “no judicial selection system is
worth a damn.”2 Against this backdrop, strident normative arguments to the effect
that one leading mechanism of judicial selection is plainly superior to another
should be viewed with skepticism. If the truth were that one-sided, odds are that we
would be much closer to consensus than we are.
Empirical scholarship and its critique have tended to puncture such stridency,
and in that way, confirm accepted wisdom. One illustrative point and counterpoint:
In my 2002 article, intemperately titled “Why Judicial Elections Stink,” I cherrypicked data to support the proposition that up to 80% of those who vote in judicial
elections, cannot identify the candidates for whom they voted, and quoted a
Michigan public official for the proposition that “[p]eople just don’t know who
they’re voting for.”3 Hall & Bonneau, however, subsequently marshaled empirical
evidence to show that this claim was overstated: in Supreme Court races, voters
1
preference candidates with judicial experience, which shows a measure of
discrimination in the choices they make.4 But on this same point, Hall & Bonneau
overstate their own, comparably strident case for judicial elections. They conclude
that “voters in state supreme court elections make fairly sophisticated candidatebased evaluations,”5 on the strength of the data point that challengers with prior
judicial experience performed “almost five percent better than their inexperienced
counterparts.“6 Setting to one side whether a differential of less than five percent on
an isolated criterion is enough to show “sophisticated candidate-based evaluations,”
they never address the extent to which experienced challengers fare better with
voters not because they are seen as more qualified, but because they better
campaigners, having prior experience running for judicial office. By way of segue to
my next point, Larry Baum and David Klein’s research suggests the possibility that
the truth lies between these extremes: voters are neither uneducable nor
sophisticated.7
2. Moving credible arguments toward the middle
“Observe due measure. Moderation is best in all things.”
--Hesiod
Insofar as empirical studies expose strident arguments for or against a given
method of judicial selection as exaggerated, the truth must lie somewhere in
between. Polemics for or against appointed and elected judiciaries implicitly regard
judicial systems as monolithic—as if what is best for Mississippi is best for
Massachusetts, what is best for the Texas Supreme Court is best for Texas trial
courts, what is best for the federal courts is best for the states, and what was best in
1789 is best today. Perhaps it is time to rethink such one-size fits all approaches.
At its core, the normative debate over judicial selection boils down to
whether it is better for judges to be independent from or accountable to the
electorate. Proceeding from the premise that judges should follow applicable law,
appointments proponents often employ a naïve version of the legal model of judicial
decision-making, and argue that judges who are not independent from the
electorate will disregard the law and yield to the electorate’s preferences. Election
proponents, in contrast, often employ a comparably naïve version of the attitudinal
model of judicial decision-making, and argue either: that judges will disregard the
law and substitute their policy preferences unless they are accountable to the
electorate, or that law is essentially irrelevant, and that elections are democracyenhancing because they incentivize judges to implement the public’s policy
preferences instead of their own.
A recent wave of empirical scholarship, however, has debunked this stark
law-policy dichotomy.8 It has shown that law and policy are inextricably
intertwined, and that judges are subject to a host of legal and extralegal influences
that vary by issue, by court, and by state. These findings defeat arguments for or
against judicial elections premised on false assumptions that law means nothing or
2
everything to independent judges, and move supportable normative arguments to
the middle. In that space, there is room to argue that independence from electoral
accountability serves other ends for judges whose decisions are influenced by law
and public policy: it insulates them from political pressure to abort or contort due
process; it liberates them to follow the law in more flexible, post-positivist ways;
and it enables them to reach results they regard as “just,” in a pragmatic, factsensitive sense of the term. But in context, each of these claims is vulnerable to
viable counter-arguments.
Criminal sentencing offers a useful illustration, because it is an area where
traditional “rule of law” justifications for independence from electoral accountability
are particularly difficult to sustain, given the paucity of applicable “law” and the
breadth of judicial discretion. The data show that judges impose stiffer criminal
sentences when their elections are impending.9 Proponents of judicial elections
who reflexively hail this as a shining example of democracy in action miss an
essential point: yes, criminal sentencing is a form of policy-making, but it is fact
sensitive policy-making that requires a measure of expertise and familiarity with the
admissible evidence and the dictates of due process the electorate lacks. The more
this individualized process is disrupted by a judge’s fear that she will lose her job if
she does not acquiesce to the public’s generalized preferences for harsher
sentences, the greater the threat to justice and due process in individual cases. On
the other hand, one can rejoin that electoral accountability does not undermine a
pragmatic, fact-sensitive pursuit of justice, but merely establishes the politically
acceptable limits of judicial discretion. If judges are free to disregard those limits, it
can breed disrespect for the courts and threaten their legitimacy.
Empirical study informs the normative debate, and moves the best
arguments to the middle, but does not resolve them. That is because the debate
requires participants to play “Would You Rather” on a grand scale, and make a
choice between unattractive alternatives: Would you rather risk injustice at the
hands of judges under ham-handed political pressure? Or injustice at the hands of
rogue judges run amok? Different states, facing different problems at different
times, have answered this question in different ways. Perhaps that is how it should
be. Such a conclusion does not come easily to disputants in the heat of battle, who
are convinced by the rightness of their polarized world-view and whose
interpretation of the “science” impels them to universalize. But just as the
attitudinal model teaches that a judge’s interpretation of law is influenced by her
ideological predilections, the same may be said for an academic’s interpretation of
data.10
Moving the normative debate toward the middle also underscores a need to
focus more attention on re-selection than initial selection. One can debate whether
a judicial candidate’s future independence is constrained in undesirable ways if, at
the point of initial selection, she pre-commits to decide future issues in specified
ways, or accepts campaign support from interested observers that engenders a
3
“debt of gratitude.” But the issue is far more acute at the point of re-selection, for a
judge who is put at risk of losing her job if she makes an unpopular choice.
3. Marshaling Support From “Public” Opinion
“You keep using that word. I do not think it means what you think it
means.”
--Inigo Montoya, The Princess Bride
One of the few areas of consensus among the bench, bar and academy in the
judicial selection debate, is that preserving public confidence in the courts is a
relevant concern. Judges, lawyers and law professors have argued (often without
empirical support) that public confidence in the courts is threatened by partisan
judicial elections, escalating interest group spending in judicial campaigns, and a
Supreme Court decision entitling judicial candidates to announce their views on the
issues they will decide as judges. Jim Gibson and others, however, have produced
survey data that contradict such claims.
The data show that subjecting judges to contested elections does not
undermine the judiciary’s legitimacy in the general public’s mind. Gibson’s data lead
him to go further still and conclude that elections are legitimacy enhancing.11
Without disputing that when asked, respondents prefer elected judiciaries, I would
add only that the unelected federal judiciary and elected state judiciaries receive
significant and comparable levels of public support.12 In other words, the public’s
preference for elected judiciaries in principle does not necessarily translate into
measurable differences in public support between one court system and another
that can be attributed to differences in method of selection.
More importantly, however, I want to suggest the possibility that the legal
establishment’s legitimacy arguments are driven less by ignorance and stupidity,
than by experience with and concern for a different kind of “public.” Judges and
lawyers are less concerned with the “public” defined as whomever researches can
coax to participate in a survey, than with the “public” that serves as jurors,
witnesses, litigants and others immediately affected by what judges do. Data
convincingly show that this latter “public” has less confidence in the courts than the
more general public.13 Additional survey data are needed to corroborate the point,
but it seems reasonable to speculate that participants in legal proceedings with skin
in the game will be more concerned than the general public that the judge’s
impartiality might be impaired by an impending election, the judge’s campaign
contributors, or the positions the judge took from the campaign stump. As a
consequence, normative arguments on judicial selection that rely on survey data
concerning public support, should take into account empirical research by Tom
Tyler and others, who focus on legitimacy from the perspective of court users.14
4. Acknowledging the limits of quantitative, empirical analysis
4
"Not everything that can be counted counts, and not everything that
counts can be counted."
- Albert Einstein
There is a difference of opinion on the value of questions that do not lend
themselves to empirical verification that approaches a disciplinary divide. For some,
arguments grounded in non-falsifiable premises are dismissed because they cannot
be verified. But that doesn’t mean those arguments are wrong, or unworthy of
discussion. For example, post-positivist conceptions of judicial decision-making
posit that “law” is elastic enough to accommodate differing interpretations at the
margins, and that independent judges follow the law as they understand it to be
written, even if different judges with different ideologies construe the law
differently. This argument reconciles the attitudinal argument that judges are
influenced by their ideologies, and the legalist argument that judges follow the law.
Moreover, it counters claims that judges who are independent from electoral
accountability disregard the law.
There is no way to model for “law” so flexibly defined, and no way to verify
judges’ sincerity. Echoing the sentiments of Einstein, however, Steve Burbank has
observed that he “prefer[s] the messiness of lived experience to the tidiness of
unrealistically parsimonious models.”15 Understanding the messiness of lived
experience is ultimately a multi-disciplinary undertaking that political scientists and
law professors are ill-equipped to address without recourse to psychology, history,
sociology and economics.
Moreover, empirical research informs but is no substitute for theory. Judicial
elections lend themselves to rational choice analysis hybridized to address
distinctive features of the judicial role. A more fully developed positive theory of
judicial selection, in turn, can inform normative theory, but to date, theoretical work
in the judicial selection arena has been limited.
More troubling than the paucity of normative theoretical scholarship, is the
possibility that quantitative empirical scholars see little value in such work. In his
chapter for a book I edited, David Pozen, associate professor of law at Columbia
University, replied to a chapter written by Melinda Gann Hall, with a measured
critique of judicial elections grounded in democratic theory. The basic thrust of
Pozen’s approach was to accept Hall’s empirical claims (although he did sometimes
quarrel with conclusions she drew from her data), take them to their logical
conclusion—that elected judges will be more responsive to majoritarian
preferences—and argue that in the context of constitutional interpretation, such
results ultimately impoverish rather than enhance democracy.16 The law professor
who refereed the chapter for the University Press that published the book liked the
Pozen chapter, and post-publication reviews singled the chapter out for praise. But
the political scientist referee wanted it cut: “Instead of addressing the empirical
evidence and discussing how the evidence may not necessarily lead to the
normative conclusions reached by Hall,” the referee observed, “Pozen simply
5
ignores it and spends considerable time discussing the work of political theorists,
who have nothing to say about the empirical reality of judicial elections.”
For quantitative empirical scholars interested in the normative implications
of their work, it is critical that they invite and engage normative theory, rather than
dismiss it. Theorists lack empiricists’ expertise to generate useful data, and
empiricists lack theorists’ expertise to evaluate the normative implications of that
data. There is room at the table for both.
John F. Kimberling Professor of Law, Indiana University Maurer School of Law
Quoted in Lee Epstein, Shedding Empirical Light on Judicial Selection, 74 Mo. L. Rev. 563, 563
(2009).
3 Charles Gardner Geyh, Why Judicial Elections Stink, 64 OH. ST. L. REV. 43, 54 (2003).
4 Chris W. Bonneau & Melinda Gann Hall, IN DEFENSE OF JUDICIAL ELECTIONS 98 (2009).
5 Id. at 133.
6 Id. at 98.
7 Lawrence Baum & David Klein, Voter Responses to High-Visibility Judicial Campaigns, in RUNNING FOR
JUDGE: THE RISING POLITICAL, FINANCIAL, AND LEGAL STAKES OF JUDICIAL ELECTIONS 204, 205-06 (Matthew J.
Streb ed. 2007).
8 WHAT’S LAW GOT TO DO WITH IT? WHAT JUDGES DO, WHY THEY DO IT, AND WHAT’S AT STAKE 41, 53 (Charles
Gardner Geyh, Ed. 2011)
9 Paul Brace & Brent Boyea, Judicial Selection Methods and Capital Punishment in the American States,
in RUNNING FOR JUDGE, supra note 7, at 186, 193-94, 197.
10 Robert MacCoun, Biases in the Use and Interpretation of Research Results, 49 ANNUAL REVIEW OF
PSYCHOLOGY 259 (1998).
11 JAMES GIBSON, ELECTING JUDGES: THE SURPRISING EFFECTS OF CAMPAIGNING ON JUDICIAL LEGITIMACY (2012).
12 Belden Russonello & Stewart, Access to Justice and Constitutional Rights Versus Political Pressure:
Defining the Battle for the Courts (Sept. 2005), in JUSTICE AT STAKE CAMPAIGN, SPEAK TO AMERICAN
VALUES: A HANDBOOK FOR WINNING THE DEBATE FOR FAIR AND IMPARTIAL COURTS app. at 17 (2006).
http://www.georgiacourts.org/aoc/r_p_docs/speaktoamericasvalues2.pdf.
13 DAVID B. ROTTMAN, RANDALL HANSEN, NICOLE MOTT, & LYNN GRIMES, PERCEPTIONS OF THE COURTS IN YOUR
COMMUNITY: THE INFLUENCE OF EXPERIENCE, RACE AND ETHNICITY, FINAL REPORT 60 (2003), available at
http://www.ncjrs.gov/pdffiles1/nij/grants/201302.pdf.
14 TOM TYLER, ROBERT BOECKMANN, HEATHER SMITH, & YUEN HUO, SOCIAL JUSTICE IN A DIVERSE SOCIETY 82-83
(1997).
15 Stephen B. Burbank, On the Study of Judicial Behaviors: Of Law, Politics, Science, and Humility, in
WHAT’S LAW GOT TO DO WITH IT?, supra note 8 at 41.
16 David Pozen, Are Judicial Elections Democracy-Enhancing?, in WHAT’S LAW GOT TO DO WITH IT?, supra
note 8 at 248.
1
2
6