Data Analytics
and the
Illinois Supreme Court
Kane County Bar Association Appellate Practice Committee
July 10, 2015
Kirk C. Jenkins
Sedgwick LLP
The Value of Data Analytics
 Distinguishing Appellate Law from Trial Law
• Persuading a Panel, not a Single Decision-Maker
 The Science of Group Decision-Making
• Statistical Analysis of Judicial Behavior:
Jurimetrics – dates back to 1941
• Game Theory
• Microeconomic Theory
• Psychological Group Theories
• Restraints on Voting Individual Preferences
• Logistic Regression Models of Court Decision-Making
The Value of Data Analytics II
 Bringing Precision to Conventional Wisdom
• “The Supreme Court Doesn’t Grant to Affirm”
• “The Supreme Court Never Reviews Rule 23 Orders or
Unanimous Decisions”
• “The Court Doesn’t Care What District the Decision Comes From”
• “Appellate decisions are getting longer”
• This pair or that pair of Justices “always votes together”
• Oral argument tells you nothing about the decision – the Justices
are playing devil’s advocate, or talking to each other
The Value of Data Analytics III
“The better that judges are understood, the more
effective lawyers will be both in litigating cases
and, as important, in predicting the outcome of
cases, thus enabling litigation to be avoided or
cases settled at an early stage.”
Lee Epstein, William M. Landes & Judge Richard A. Posner, The
Behavior of Federal Judges: A Theoretical and Empirical Study of
Rational Choice, (Cambridge: Harvard Univ. Press, 2013).
The Supreme Court Data Library
 629 civil cases, 2000-2014
 More than 36,000 data points in the library
 Case Identifiers:
• Case Name
• Docket Number
 Before the Lower Courts:
•
•
•
•
Appellate Court
Basis for Appellate Jurisdiction
Appeal Taken from Summary Judgment (Y/N)
Area of the Law
The Supreme Court Data Library II
•
•
•
•
•
Philosophical Coding of Appellate Court decision (C/L)
Dissent before Appellate Court (Y/N)
Appellate Court decision published (Y/N)
Circuit Court or Board of original jurisdiction
Trial Judge
 The Decision Process Before the Supreme Court:
•
•
•
•
•
Date of Oral Argument
Date of Decision
Days under Submission
Amici
Result & Vote
The Supreme Court Data Library III
 The Opinions
• Length of Majority Opinion
• Length of Special Concurrences
• Length of Dissents
 Justice-by-Justice Data
•
•
•
•
Vote
Opinion (M/SC/D/joined SC or D)
Philosophical Coding of Vote
Recused (Y/N)
The Supreme Court Data Library IV
 Oral Arguments
• All Counsel Presenting Argument
• Justice-by-Justice, First Question (Appellant/Appellee/Rebuttal)
• Justice-by-Justice, Number and Order of Questions
Where the Docket Comes From
Leading Subjects, 2000-2014
Tort
Total Cases
142
Civil Procedure
70
Government/Administrative
61
Constitutional
57
Domestic Relations
44
Insurance
43
Workers’ Compensation
27
Which Circuit Courts?
10
9
4
2
1
6
Originating County,
2000-2014
1 Cook
7
3
8
5
Percentage of
Civil Docket
44.9
2 Du Page
6.2
3 Sangamon
5.4
4 Lake
4.9
5 St. Clair
3.9
6 Will
3.6
7 Champaign
2.1
8 Madison
2.0
9 Winnebago
1.1
10 McHenry
0.7
Are Summary Judgments Easier?
Year
Percentage of Civil
Docket from SJ
2014
25.9
2013
29.4
2012
22.5
2011
15.8
2010
20.0
2009
26.8
2008
19.5
2007
26.8
2006
14.9
2005
17.4
2004
27.8
Do Dissents Help?
Year
Percentage of Docket
Dissent Below
2014
14.8
2013
29.4
2012
30.8
2011
26.3
2010
36.4
2009
28.2
2008
29.3
2007
36.6
2006
25.5
2005
26.8
2004
21.7
Time Under Submission
 Lag Time between Argument and Decision
UNANIMOUS DECISIONS
2010
129.6
2014
100.7
DIVIDED DECISIONS
2010
203.9
2014
193.8
coming more quickly
Frequency of Unanimous Decisions
Year
Percentage of
Civil Docket
2014
71.4
2013
58.8
2012
55.0
2011
76.3
2010
72.7
2009
80.5
2008
70.7
2007
80.5
2006
57.4
2005
80.4
2004
70.4
Are Dissents Sharper?
Year
Percentage 0-1
Dissenters
Percentage 2-3
Dissenters
2014
76.2
23.8
2013
79.4
20.6
2012
72.5
27.5
2011
86.8
13.2
2010
81.8
18.2
2009
90.2
9.8
2008
80.5
19.5
2007
90.2
9.8
2006
83.0
17.0
2005
95.7
4.3
2004
77.8
22.2
Reversal Rates by Appellate District
1st, 1 1st, 2 1st, 3 1st, 4 1st, 5 1st, 6 2
3
4
5
2014
100
80
75
83
33
73
81
57
50
82
2013
100
73
50
67
60
67
59
60
47
82
2012
63
90
50
30
80
77
58
75
30
79
2011
60
80
55
22
80
70
50
64
46
86
2010
27
83
88
33
50
70
57
70
50
92
2009
25
78
75
57
60
50
61
53
56
91
2008
13
62
63
44
50
40
50
46
56
73
2007
29
62
50
38
60
27
48
39
46
77
2006
50
33
50
50
43
29
50
36
46
73
2005
67
25
30
50
38
50
65
54
47
71
2004
64
0
22
67
38
67
70
54
59
52
What Kinds of Cases are Granted?
Area of Law
2010-2014
Conservative
Appellate Court
Decisions
Liberal Appellate
Court Decisions
Tort
30.8
69.2
Civil Procedure
50
50
Domestic Relations
53.3
46.7
Constitutional Law
50
50
Government and Administrative
45.5
54.5
Insurance
22.2
77.8
Workers’ Comp
50
50
Differing Reversal Rates?
Area of Law
2010-2014
Conservative
Appellate Court
Decisions
Liberal
Appellate Court
Decisions
Tort
41.7
69.2
Civil Procedure
54.5
72.7
Domestic Relations
62.5
50.0
Constitutional Law
14.3
71.4
Government and Administrative
60.0
83.3
Insurance
100.0
57.1
Workers’ Comp
100.0
50.0
Voting Dynamics – The Court’s Center
 Agreement Rates (Non-Unanimous Civil Cases, 2014)
2012
2013
2014
Garman-Thomas
81.8
84.6
83.3
Garman-Karmeier
75.0
73.8
83.3
Thomas-Karmeier
85.3
82.1
100.0
Voting Dynamics – Counting to 4
 The Fourth Vote
2012
2013
2014
Garman-Burke
72.2
71.4
65.7
Garman-Theis
74.1
77.5
79.4
Thomas-Burke
67.6
66.7
62.5
Thomas-Theis
64.0
67.6
67.7
Karmeier-Burke
75.0
73.8
74.3
Karmeier-Theis
66.7
70.0
70.6
Oral Argument - Which Side Gets
More Questions?
Year
Appellants
Appellees
2008
418
444
2009
765
497
2010
639
445
2011
635
387
2012
589
609
2013
524
316
2014
458
328
Total
4028
3026
Hot & Cold Courts
81
Most questions
in a single civil
argument
In re Sophia G.L.,
2008
57
44
Most questions
to an appellant
Most questions
to an appellee
In re Sophia G.L.
(Reversed)
Secura Insurance
Company v. Illinois
Farmers Insurance
Co., 2009 (Reversed)
8
Fewest questions
in a single
argument (both
sides represented)
Russell v. SNFA, 2013
Are More Questions a Bad Sign? –
Avg. Questions per Argument
Year
Winning
Appellants
Losing
Appellants
Winning
Appellees
Losing
Appellees
2008
16.53
18.89
12.22
22.27
2009
19.93
20.64
11.00
13.93
2010
17.39
21.73
11.93
14.78
2011
14.72
30.56
7.83
14.48
2012
15.00
17.11
9.44
18.07
2013
13.63
17.67
7.87
10.42
2014
13.18
23.40
10.10
13.35
Total
15.96
20.07
9.83
15.29
Does It Matter if the Court is Split?
Year
Appellants
Unanimous
Appellants Not
Unanimous
Appellees
Unanimous
Appellees Not
Unanimous
2008
18.78
13.33
18.72
17.83
2009
19.66
21.57
16.47
12.86
2010
19.50
19.00
14.17
11.67
2011
17.64
16.44
9.78
14.56
2012
15.00
16.28
15.76
15.89
2013
13.65
17.21
9.45
9.07
2014
17.05
16.83
11.84
17.17
Total
17.53
17.17
13.75
13.75
Does It Matter if the Court is
Closely Split?
Year
Appellants
Unanimous
Appellants 3
Dissenters
Appellees
Unanimous
Appellees 3
Dissenters
2008
18.78
2.00
18.72
22.00
2009
19.66
41.00
16.47
15.00
2010
19.50
23.50
14.17
8.50
2011
17.64
22.50
9.78
22.50
2012
15.00
9.75
15.76
11.00
2013
13.65
24.00
9.45
3.00
2014
17.05
19.50
11.84
22.00
Overall
17.53
18.08
13.75
14.62
Is the First Question the Majority Author?
Percentage of Arguments
First Question from
Author of Majority
Winning Appellant
25.17
Losing Appellant
24.69
Winning Appellee
19.75
Losing Appellee
28.06
Who Asks the Most Questions?
Thomas
Freeman
Kilbride
2,223
701
604
Burke
Fitzgerald
Theis
909
678
553
Garman
Karmeier
897
643
Average Questions per Argument
Appellant
Appellee
Rebuttal
Thomas
3.90
4.42
1.35
Burke
1.91
1.69
0.42
Garman
1.56
1.97
0.32
Freeman
1.79
1.07
0.23
Fitzgerald
3.72
2.75
0.67
Karmeier
1.06
1.30
0.41
Kilbride
0.95
1.19
0.55
Theis
2.20
1.49
0.53
How Often is each Justice the
First Questioner?
Appellant
Appellee
Rebuttal
Thomas
33.48
39.13
29.57
Burke
18.58
12.83
4.87
Garman
15.02
15.45
6.00
Freeman
22.37
11.84
2.19
Fitzgerald
12.63
12.63
17.89
Karmeier
5.60
9.91
9.91
Kilbride
2.67
4.44
6.22
11.45
7.63
10.69
Theis
Does the Result Matter?
Appellant
Wins
Appellant
Loses
Appellee
Wins
Appellee
Loses
Thomas
4.50
6.49
2.58
5.77
Burke
1.88
3.23
1.53
2.02
Garman
1.75
2.01
1.20
2.46
Freeman
2.25
1.70
1.19
1.35
Fitzgerald
4.69
4.09
2.71
4.10
Karmeier
1.33
1.72
1.43
1.28
Kilbride
1.41
1.56
1.08
1.29
Theis
2.04
3.81
1.38
1.64
Justice Burke’s
Question Patterns
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
1.50
--
--
4.50
Appellant – RR
1.40
2.88
14.00
--
Appellant – AA
2.46
6.64
None
--
Appellant – AR
1.17
--
--
6.00
Appellee – RA
0.50
--
--
2.25
Appellee – RR
1.91
2.96
3.00
--
Appellee – AA
1.53
1.43
2.00
--
Appellee – AR
0.80
--
--
4.25
Justice Burke
& the First Question
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
0.50
--
--
0.50
Appellant – RR
0.20
0.41
0.00
--
Appellant – AA
0.09
0.71
None
--
Appellant – AR
0.17
--
--
0.00
Appellee – RA
0.00
--
--
0.00
Appellee – RR
0.16
0.24
0.00
--
Appellee – AA
0.16
0.29
0.00
--
Appellee – AR
0.00
--
--
0.00
Chief Justice Garman’s
Question Patterns
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
3.67
--
--
4.00
Appellant – RR
1.64
2.32
2.00
--
Appellant – AA
1.96
3.78
2.00
--
Appellant – AR
0.50
--
--
1.50
Appellee – RA
0.00
--
--
3.00
Appellee – RR
1.64
3.86
0.50
--
Appellee – AA
0.96
1.67
None
--
Appellee – AR
3.00
--
--
2.75
Chief Justice Garman
& the First Question
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
0.00
--
--
0.33
Appellant – RR
0.19
0.58
0.50
--
Appellant – AA
0.18
0.44
0.00
--
Appellant – AR
0.50
--
--
0.50
Appellee – RA
0.00
--
--
0.33
Appellee – RR
0.12
0.45
0.50
--
Appellee – AA
0.07
0.33
None
--
Appellee – AR
0.67
--
--
0.50
Justice Freeman &
the First Question
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
1.00
--
--
0.50
Appellant – RR
0.31
0.20
0.00
--
Appellant – AA
0.14
0.06
0.00
--
Appellant – AR
0.00
--
--
0.29
Appellee – RA
None
--
--
0.20
Appellee – RR
0.14
0.00
0.00
--
Appellee – AA
0.09
0.14
0.00
--
Appellee – AR
0.00
--
--
0.57
Justice Kilbride’s
Question Patterns
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
1.33
--
--
2.20
Appellant – RR
1.28
2.00
0.00
--
Appellant – AA
1.71
0.71
4.00
--
Appellant – AR
0.50
--
--
2.67
Appellee – RA
0.50
--
--
1.08
Appellee – RR
1.07
3.93
0.00
--
Appellee – AA
1.08
0.75
2.00
--
Appellee – AR
2.25
--
--
0.78
Justice Kilbride
& the First Question
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
0.00
--
--
0.10
Appellant – RR
0.10
0.21
0.0
--
Appellant – AA
0.06
0
0
--
Appellant – AR
0.00
--
--
0.11
Appellee – RA
0.00
--
--
0.08
Appellee – RR
0.06
0.07
0.00
--
Appellee – AA
0.02
0.00
0.00
--
Appellee – AR
0.00
--
--
0.00
Justice Thomas’s
Question Patterns
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
6.00
--
--
16.30
Appellant – RR
3.98
4.86
0
--
Appellant – AA
6.44
6.85
7.00
--
Appellant – AR
2.00
--
--
4.50
Appellee – RA
0.00
--
--
1.00
Appellee – RR
5.70
7.18
2.50
--
Appellee – AA
2.61
2.38
1.00
--
Appellee – AR
2.00
--
--
5.50
Justice Thomas
& the First Question
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
0.00
--
--
1.33
Appellant – RR
0.49
0.72
0.00
--
Appellant – AA
0.76
0.77
1.00
--
Appellant – AR
2.00
--
--
1.00
Appellee – RA
0.00
--
--
0.00
Appellee – RR
0.46
0.77
0.50
--
Appellee – AA
0.36
0.15
1.00
--
Appellee – AR
1.00
--
--
2.00
Justice Karmeier’s
Question Patterns
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
3.33
--
--
0.00
Appellant – RR
1.04
3.11
0.00
--
Appellant – AA
1.03
4.27
6.00
--
Appellant – AR
0.67
--
--
1.20
Appellee – RA
0.00
--
--
0.00
Appellee – RR
1.25
1.70
1.00
--
Appellee – AA
0.97
4.00
1.50
--
Appellee – AR
0.00
--
--
0.00
Justice Karmeier
& the First Question
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
0.67
--
--
0.00
Appellant – RR
0.12
0.26
0.00
--
Appellant – AA
0.12
0.53
0.00
--
Appellant – AR
0.33
--
--
0.00
Appellee – RA
0.00
--
--
0.00
Appellee – RR
0.03
0.15
0.00
--
Appellee – AA
0.17
0.33
0.50
--
Appellee – AR
0.00
--
--
0.00
Justice Theis’s
Question Patterns
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
3.75
--
--
5.00
Appellant – RR
1.74
2.64
5.00
--
Appellant – AA
2.49
14.33
0.00
--
Appellant – AR
0.00
--
--
0.00
Appellee – RA
3.00
--
--
1.33
Appellee – RR
1.37
2.14
10.00
--
Appellee – AA
1.28
1.67
1.00
--
Appellee – AR
5.00
--
--
0.00
Justice Theis
& the First Question
Not Writing
Majority
Concurrence
Dissent
Appellant – RA
0.00
--
--
0.33
Appellant – RR
0.15
0.50
0.00
--
Appellant – AA
0.23
0.67
0.00
--
Appellant – AR
0.00
--
--
0.00
Appellee – RA
0.00
--
--
0.00
Appellee – RR
0.03
0.07
0.00
--
Appellee – AA
0.10
0.50
0.00
--
Appellee – AR
0.00
--
--
0.00
Applying Our Conclusions
 In re Pension Reform Litigation, 2015
Appellant
Appellee
Rebuttal
Burke
0
0
0
Kilbride
0
0
0
Freeman
0
0
0
Garman
1
2F
1
Thomas
11F
0
6
Karmeier
0
0
1F
Theis
0
0
0
Result . . .
7-0
Affirmance
Justice Karmeier for the Court
Applying Our Conclusions II
 Leetaru v. The Board of Trustees of Univ. of Illinois, 2015
– 210 days under submission
Appellant
Appellee
Rebuttal
3F
0
0
Kilbride
0
0
0
Freeman
0
0
0
Garman
1
2
1F
Thomas
3
4F
0
Karmeier
2
9
1
Theis
0
0
0
Burke
Result . . .
4-3
Reversal
Justice Karmeier for the Court
Justice Burke dissenting
Applying Our Conclusions III
 Skokie Castings v. Illinois Insurance Guar. Fund, 2013 –
332 days under submission
Appellant
Appellee
Rebuttal
Burke
4
2
3
Kilbride
6
0
0
Freeman
0
0
0
Garman
2
1
0
Thomas
3F
4
1F
Karmeier
2
2
2
Theis
0
1F
0
Result …
5-2
Affirmance
Justice Karmeier for the Court
Justice Thomas dissenting
Justice Kilbride dissenting
Lessons from the Data
 A side asked significantly more questions is in trouble –
both overall, and with each Justice
 The first questioner may be writing the majority
 Chief Justice Garman, Justices Burke, Karmeier, Theis
and to a degree Thomas – more active if writing, more
likely to be the first question
 Treating Questions as a Tentative Opinion
Sedgwick’s Appellate Task Force
Hall R. Marston
Douglas L. Collodel
Michael M. Walsh
Matthew A. Reed
Kirk C. Jenkins,
Chair
(also California)
Agelo L. Reppas
S. Vance Wittie
Robert C. Weill
Erin E. Dardis
Peter C. Condron
Aaron F. Mandel
Questions?
Kirk C. Jenkins
One N. Wacker Drive, #4200
Chicago, IL 60606
Tel: (312) 641-9050
Kirk.Jenkins@SedgwickLaw.com
Google Plus:
https://plus.google.com/+KirkJenkins/posts
LinkedIn:
https://www.linkedin.com/in/kirkcjenkins
Twitter: @KirkCJenkins
@ISCReview
Blogs:
Appellate Strategist
Illinois Supreme Court Review