Judicial Decision under Ambiguity and Predictive
Justice
Sébastien Massoni1
1 BETA
Vincent Teixeira1
- Université de Lorraine
L2 - Défis Economiques du XXIe siècle
March 23, 2023
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
1 / 42
Predictive Justice
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
2 / 42
Predictive Justice
COMPASS algorithm was used in Wisconsin to estimate risk
assessment in sentencing despite a non-disclosed algorithm, biased
toward minority, not developed for sentencing and predicting at the
group level (State v. Loomis, Kehl et al., 2017).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
2 / 42
Predictive Justice
Natural Language Processing and Machine Learning algorithms reach
79% of accuracy on predicting the outcome of cases tried by the
European Court of Human Rights based solely on textual content
(Aletras et al., 2016).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
2 / 42
Predictive Justice
A.I. ’Prosecutor’ that can press charges with more than 97% accuracy
in China.
It can identify and press charges for Shanghai’s 8 most common
crimes (e.g. credit card fraud, running a gambling operation, etc.).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
2 / 42
LegalTechs and Predictive Justice
Definitions
LegalTech: Use of technology and software to provide legal services
and support the legal industry; often associated with technology
startups disrupting the practice of law.
Predictive justice: computer tool used to carry out statistical
analyses based on a big data extracted from case law, i.e. court
decisions previously rendered.
Predictive justice algorithms might:
Process behavioral data to assess individual behaviors (e.g. risk of
recidivism);
Analyze legal data to produce decision models and anticipate court
decisions (similar cases, effective legal arguments, assessment of
chances, average amount perceived or procedure duration).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
3 / 42
Characteristics of the Law Market
Credence goods market with law agents (lawyers, judges, etc.) and
litigants.
Before digitalization
After digitalization
Regulated profession
Break of the oligopoly
Noisy information
(reputation, advertisement)
More objective information
Transaction costs
Potential decrease of costs
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
4 / 42
LegalTechs and Information
LegalTechs brought 3 types of innovation:
Intermediation - relation between justice and its actors;
Processing and provision of data;
Artificial Intelligence (A.I).
The two first types of innovation have a common point: they bring new
information.
We design two studies to understand how this new source of information
might change behaviors in ambiguous judicial decisions.
We use real judicial cases and a predictive justice tool: Case Law Analytics.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
5 / 42
Case Law Analytics
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
6 / 42
Case Law Analytics
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
6 / 42
Case Law Analytics
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
6 / 42
Case Law Analytics
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
6 / 42
Case Law Analytics
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
6 / 42
Risk and Ambiguity
Risk: Known probabilities
Ambiguity: Unknown probabilities
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
7 / 42
Risk and Ambiguity
Risk: Known probabilities
Ambiguity: Unknown probabilities
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
7 / 42
About Ambiguity
It requires new models: e.g. multiple priors, probability weighting.
Two main measures of ambiguity attitudes (Abdellaoui et al., 2011):
Ambiguity aversion / pessimism;
Likelihood insensitivity / perceived level of ambiguity.
Ambiguity aversion is generally observed but not universal (e.g.
Kocher et al., 2018, for losses and lower likelihoods).
Two questions related to our study:
Sources of ambiguity;
Value of information.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
8 / 42
Sources of Ambiguity: Artificial and Natural Events
Sources of ambiguity matter (Tversky & Fox, 1995; Abdellaoui et al.,
2011).
Artificial events (e.g. lotteries and urns)
Urns allow for a control of likelihood beliefs;
But their external validity is questioned (Heath & Tversky, 1991;
Camerer & Weber, 1992).
Natural events (e.g. weather and stock markets predictions)
Difficulties to estimate likelihood beliefs may prevent the identification
of ambiguity attitudes;
Ambiguity attitudes are closer to rationality (Li et al., 2018).
Measurements without likelihood beliefs (Baillon et al., 2018)
Ambiguity aversion and ambiguity generated insensitivity (perception of
ambiguity);
Indexes are valid for artificial or natural events and under many
ambiguity theories.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
9 / 42
Information in Decision under Ambiguity
Value of information
Ambiguity aversion increases the value of information (Snow, 2010;
Attanasi & Montesano, 2012; Kops & Pasichnichenko, 2020);
But under-valuation of information not resolving ambiguity (Ambuehl
& Li, 2018; Li, 2020).
Beliefs updating and ambiguous information
Failure of Bayesian updating (Epstein & Halevy, 2019);
Insensitivity or ambiguity aversion in information accuracy (Liang,
2021; Shishkin & Ortoleva, 2021).
Consequentialism and dynamic consistency
Ambiguity aversion increases violation of dynamic consistency but not
consequentialism (Dominiak et al., 2012; Bleichrodt et al., 2018;
Galanis, 2021).
Information avoidance is common for real-life decisions
Health, finance, managerial decision (see Loewenstein’s works).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
10 / 42
Judicial Decision and Natural Ambiguity
Judicial decisions offer a natural source of ambiguity.
We take real cases that are already judged by a court.
We then have a situation without artificial ambiguity.
We can provide objective information through predictive justice.
We have different situations: complete ambiguity, partial ambiguity,
risk and similarity.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
11 / 42
Study 1: Ambiguity Attitudes for Judicial Events
We want to compare behaviors under artificial and natural events
(within-subject).
For three additional types of information (between-subject):
[0; 100]: C-Ambiguity (baseline);
link
p1 ; p2 ; p3 : Risk with probability distribution;
[pi ; pi ]: P-Ambiguity with confidence index;
Similarity with distribution of close cases.
Massoni & Teixeira
Ambiguity and Predictive Justice
link
link
link
L2 Défis Economiques
12 / 42
Types of Ambiguity
Decisions under natural events are all ambiguous: risk information
leads to probabilistic ambiguity due to the uncertainty of the
predictive algorithm.
While decisions under artificial events with risk information is a
standard risky choice.
Similarity can be interpreted within the case-based framework (Gilboa
& Schmeidler, 1995, 2001).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
13 / 42
Stimuli
Subjects will have to take two types of decision: 3-color Ellsberg urns
and 3 amounts of condemnation in a divorce situation.
We elicit the certainty equivalent by probability matching for each
case (cf. Baillon et al., 2018).
We will measure attitudes toward ambiguity through 2 indexes:
ambiguity aversion;
a-insensitivity (level of ambiguity).
They will all face C-Ambiguity.
By providing additional information we will implement Risk,
P-Ambiguity and Similarity.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
14 / 42
Matching Probability
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
15 / 42
Matching Probability
Three mutually exclusive and exhaustive non-null events E1 , E2 and
E3 where Ei is a single event and Eij a composite event (Ei ∪ Ej ).
Matching probability mE of event E means that receiving X (here
e20) under event E is equivalent to receiving X with probability mE .
XE 0 ∼ XmE 0
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
15 / 42
Matching Probability
The ambiguous lottery XE 0 is evaluated by its SEU:
P (E)U (X) + (1 − P (E))U (0)
The risky lottery XmE 0 is evaluated by:
mE U (X) + (1 − mE )U (0)
For ambiguity-neutral DM: P (E) = mE :
by laws of probability mE + mE C = 1.
Empirically we often observe P (E) ̸= mE :
size and sign of deviations from laws of probability reflect ambiguity
attitudes;
e.g. ambiguity averse means mE + mE C < 1.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
16 / 42
Matching Probability
Relative elevation of MP function: extent of deviation toward ambiguity
neutrality: ambiguity aversion (seeking) for likely (unlikely) events.
Relative flatness in the middle: (in)sensitivity of the DM in discriminating
different intermediate levels of likelihood of ambiguous events.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
17 / 42
Matching Probability
We observe P (E) − mE .
With known beliefs: mE captures ambiguity attitudes controlling for
risk aversion (Dimmock et al., 2016): P − m.
With unknown beliefs, Bailon et al. (2018) methods is used:
Elicitation of mEi for 3 mutually exclusive and exhaustive non-null
events and for their composites;
Allows to estimate ambiguity attitudes irrespective of risk aversion and
subjective beliefs.
E1 , E2 , E3 , {E1 ; E2 }, {E2 ; E3 }, {E1 ; E3 }
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
18 / 42
Matching Probability
E1 , E2 , E3 , {E1 ∪ E2 }, {E1 ∪ E3 }, {E2 ∪ E3 }
Average single-event MP: ms = (mE1 + mE2 + mE3 )/3;
Average composite-event MP: mc = (mE12 + mE13 + mE23 )/3;
Under ambiguity neutrality: ms =
Massoni & Teixeira
1
3
and mc = 23 .
Ambiguity and Predictive Justice
L2 Défis Economiques
19 / 42
Ambiguity aversion: b index
Ambiguity aversion index is:
b = 1 − (ms + mc )
with:
ambiguity neutrality for b = 0;
ambiguity aversion for b ∈]0; 1];
ambiguity seeking for b ∈ [−1; 0[.
Ambiguity averse DM has low m i.e. is willing to pay a premium (in
winning probabilities) to avoid ambiguity.
Maximally ambiguity averse DM has all m = 0 and thus b = 1.
The reciprocal is true for ambiguity seeking DM.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
20 / 42
A(mbiguity-generated)-insensitivity: a index
Ambiguity-generated insensitivity index is:
1
a = 3 × ( − (mc − ms ))
3
with
Ambiguity neutrality for a = 0 (perfect discrimination between single
and composite events).
Maximal insensitivity for a = 1 (i.e. mc = ms ).
a < 0 corresponds to over-sensitivity, irrationality or noise.
The better a DM can discriminate between single and composite
events, the larger mc − ms is.
Maximal insensitivity DM treats all uncertainties as fifty-fifty.
Measure of perception of ambiguity: the more ambiguity a DM
perceives, the more the likelihood of events are perceived as one blur.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
21 / 42
Ambiguity Attitudes indexes
b index captures a motivational component on ambiguity attitudes.
a index captures a cognitive component on ambiguity perception.
The two indexes are orthogonal i.e. ambiguity may still play a role
through insensitivity if there is no aversion.
These indexes unify and extend many existing indexes without the
need of specific assumptions.
e.g. multiple priors: α =
Massoni & Teixeira
b
2a
+
1
2
Ambiguity and Predictive Justice
L2 Défis Economiques
22 / 42
Assumptions
H1 Ambiguity aversion is affected by the nature of the events:
(a) Ambiguity aversion for artificial events: bartif icial > 0
(b) Absence of ambiguity aversion for natural events: bnatural ≤ 0
(c) Difference of ambiguity aversion between natural and artificial events:
bartif icial ̸= bnatural
H2 Ambiguity aversion decreases with the content of information:
(a) bambiguity > bothers
(b) bsim > bpartial > brisk
H3 A-insensitivity is affected by the nature of the events:
(a) A-insensitivity for artificial events: aartif icial > 0
(b) A-insensitivity for natural events: anatural > 0
(c) Higher a-insensitivity for natural events: anatural > aartif icial
H4 A-insensitivity decreases with the content of information:
(a) aambiguity > aothers
(b) asim > apartial > arisk
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
23 / 42
Study 1
Experiment took place at the Laboratoire d’Economie Experimentale
de Strasbourg (LEES) in November 2021.
There were 252 subjects (gender balanced) for a total of 12 sessions
(4 by treatment):
81 subjects for Risk;
88 subjects for P-Ambiguity;
83 subjects for Similarity.
The average payoff was e17 for an average time of 1h30.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
24 / 42
Timeline of decisions
Part 1: complete ambiguity: 12 rounds (case 1: 1-6; case 2: 7-12).
Part 2: one source of information (risk, partial, similarity): 12 rounds
(same cases).
Two parts with natural events and two parts with artificial events
(pseudo-randomized for order effect).
Check for consistency: no differences for indexes between rounds 1-6
and 7-12 within a same types of ambiguity.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
25 / 42
Results: b index
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
26 / 42
Results
H1 Ambiguity aversion is affected by the nature of the events:
YES Ambiguity aversion for artificial events: bartif icial > 0
YES Absence of ambiguity aversion for natural events: bnatural ≤ 0
YES Difference of ambiguity aversion between natural and artificial events:
bartif icial ̸= bnatural
H2 Ambiguity aversion decreases with the content of information:
YES in artificial bambiguity > bothers
YES only for risk in natural bambiguity > bothers
NO bsim > bpartial > brisk
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
27 / 42
Results: a index
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
28 / 42
Results
H3 A-insensitivity is affected by the nature of the events:
YES A-insensitivity for artificial events: aartif icial > 0
YES A-insensitivity for natural events: anatural > 0
NO Higher a-insensitivity for natural events: anatural > aartif icial
H4 A-insensitivity decreases with the content of information:
YES in artificial aambiguity > aothers
YES in natural except similarity aambiguity > aothers
NO asim > apartial > arisk
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
29 / 42
Study 2: Value of Information
We want to measure how much subjects value each type of information.
Subjects face C-Ambiguity and give their matching probability.
Then, they will give they Willingness To Pay (WTP) for a source of
information.
BDM mechanism will determine their access to the information.
Subjects will face each type of information for both sources (artificial
and natural).
Due to our previous results, subjects will always face natural events
first.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
30 / 42
Types of Ambiguity
Subjects play the different types of ambiguity without knowing the
output of the BDM.
Except for WTP = 0, they are facing a double ambiguous events: on
the probability and on the outcome.
See multidimensional ambiguity (Eicheberger et al., 2015; Eliaz &
Ortoleva, 2016; Aggarwal & Mohanty, 2021).
This aspect is not in the scope of the current studies.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
31 / 42
Assumptions
H5 Under-evaluation of information not resolving ambiguity:
W T Partif icial > W T Pnatural
H6 Increase of valuation with the content of information:
W T P risk > W T P partial > W T P sim
H7 Increase of valuation with the level of ambiguity aversion:
ρ(b, W T P ) > 0
H8 Link between the level of sensitivity to ambiguity and the valuation:
ρ(a, W T P ) ̸= 0
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
32 / 42
Study 2
Experiment took place at the Laboratoire d’Economie Experimentale
de Strasbourg (LEES) in January 2021.
There were 247 subjects (gender balanced) and a total of 12 sessions
(4 by treatment):
81 subjects played Risk first;
83 subjects played P-Ambiguity first;
83 subjects played Similarity first.
The average payoff was 17 euros for an average time of 1h45.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
33 / 42
Timeline of decisions
Part 1: complete ambiguity (6 rounds), WTP for information 1,
information 1 (6 rounds same case).
Part 2: complete ambiguity (6 rounds new case), WTP for
information 2, information 2 (6 rounds same case).
Part 3: complete ambiguity (6 rounds new case), WTP for
information 3, information 3 (6 rounds same case).
Order of information’s type pseudo-randomized for order effect.
3 parts with natural events first and then 3 parts with artificial events
(not-randomized).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
34 / 42
Results: WTP
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
35 / 42
Results
H5 Under-evaluation of information not resolving ambiguity:
YES W T Partif icial > W T Pnatural
H6 Increase of valuation with the content of information:
NO W T P risk > W T P partial > W T P sim
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
36 / 42
Results: WTP and ambiguity attitudes
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
37 / 42
Results
H7 Increase of valuation with the level of ambiguity aversion:
YES only in artificial ρ(b, W T P ) > 0
H8 Link between the level of sensitivity to ambiguity and the valuation:
YES ρ(a, W T P ) ̸= 0 i.e. ρ(a, W T P ) < 0
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
38 / 42
Additional results
The two indexes of ambiguity attitudes are theoretically orthogonal
but we observe a strong correlation between a and b indexes:
for artificial events in all types of ambiguity;
but not for natural events except for P-ambiguity;
These correlations are found in both studies.
Decision time seems to be linked to lower b and a indexes:
The longer a DM think about the decision, the closer he is to
ambiguity neutrality.
Regressions analyses confirm or reinforce main and additional results.
Robustness to outliers (violation of monotonicity, irrational a, etc.).
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
39 / 42
Conclusion
H1 Ambiguity aversion is affected by the nature of the events:
YES Ambiguity aversion for artificial events: bartif icial > 0.
YES Absence of ambiguity aversion for natural events: bnatural ≤ 0.
YES Difference of ambiguity aversion between natural and artificial events:
bartif icial ̸= bnatural
H2 Ambiguity aversion decreases with the content of information:
YES in artificial bambiguity > bothers
YES only for risk in natural bambiguity > bothers
NO bsim > bpartial > brisk
H3 A-insensitivity is affected by the nature of the events:
YES A-insensitivity for artificial events: aartif icial > 0.
YES A-insensitivity for natural events: anatural > 0.
NO Higher a-insensitivity for natural events: anatural > aartif icial
H4 A-insensitivity decreases with the content of information:
YES in artificial aambiguity > aothers
YES in natural except similarity aambiguity > aothers
NO asim > apartial > arisk
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
40 / 42
Conclusion
H6 Under-evaluation of information not resolving ambiguity:
YES W T Partif icial > W T Pnatural
H7 Increase of valuation with the content of information:
NO W T P risk > W T P partial > W T P sim
H8 Increase of valuation with the level of ambiguity aversion:
YES only in artificial ρ(b, W T P ) > 0
H9 Link between the level of sensitivity to ambiguity and the valuation:
YES ρ(a, W T P ) ̸= 0 i.e. ρ(a, W T P ) < 0
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
40 / 42
Discussion
Attitudes toward ambiguity differ under natural and artificial settings.
Information provided to a complete ambiguous situation change the
behaviors but with no major difference according to the content of
information.
Values given to these information are relatively in line with attitudes.
Ambiguity aversion and ambiguity generated insensitivity are related.
Judicial decision is just a framework but coming experiment on the
impact of predictive justice on lawyers behaviors and courts resolution.
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
41 / 42
Thank you for your attention
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
42 / 42
Complete Ambiguity
Massoni & Teixeira
Go back
Ambiguity and Predictive Justice
L2 Défis Economiques
42 / 42
Risk
Go back
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
42 / 42
Partial Ambiguity
Massoni & Teixeira
Go back
Ambiguity and Predictive Justice
L2 Défis Economiques
42 / 42
Similarity
Go back
Massoni & Teixeira
Ambiguity and Predictive Justice
L2 Défis Economiques
42 / 42