www.nature.com/scientificreports
OPEN
Trust, trustworthiness and AI
governance
Christian Lahusen 1,4, Martino Maggetti 2,4 & Marija Slavkovik 3,4*
An emerging issue in AI alignment is the use of artificial intelligence (AI) by public authorities, and
specifically the integration of algorithmic decision-making (ADM) into core state functions. In this
context, the alignment of AI with the values related to the notions of trust and trustworthiness
constitutes a particularly sensitive problem from a theoretical, empirical, and normative perspective.
In this paper, we offer an interdisciplinary overview of the scholarship on trust in sociology, political
science, and computer science anchored in artificial intelligence. On this basis, we argue that only
a coherent and comprehensive interdisciplinary approach making sense of the different properties
attributed to trust and trustworthiness can convey a proper understanding of complex watchful trust
dynamics in a socio-technical context. Ensuring the trustworthiness of AI-Governance ultimately
requires an understanding of how to combine trust-related values while addressing machines, humans
and institutions at the same time. We offer a road-map of the steps that could be taken to address the
challenges identified.
Artificial intelligence (AI) is now widely recognised as a socio-technical phenomenon that creates unprecedented
opportunities while also possibly posing existential ­risks1,2. Overly positive or negative outlooks on the impact
of AI tend to overestimate both the potential as well as the threat of artificial intelligence, but they are correct
in highlighting the progressive incorporation of AI-based systems into all aspects of societal l­ ife3. This observation also applies to public authorities, which notably adopted AI because they see the merits of investing into
technological innovation in order to increase the efficiency, effectiveness, and quality of services. Automation
of decisions with the help of AI has thus been increasingly, and sometimes invisibly, introduced into core state
activities such as public policy decision-making, internal management, and public service delivery in a wide
range of ­areas4,5.
The relationships between public authorities and citizens are particularly affected by the growing reliance
on AI automation, namely algorithmic decision-making (ADM)—software tools that use AI to aid and to make
automated decisions. ADM is increasingly used to manage citizens’ applications and requests, identify personalized services, predict risks in regard to service provision, specify potential sanctions, and communicate decisions.
ADM systems are thus having concrete effects in several domains, especially in—but not limited to—high-income
countries. For instance, automation is being applied by public authorities to identify the eligibility for children’s
­allowances6,7, to select available employment offers for ­jobseekers8, to calculate social b
­ enefits9, to detect tax
­fraud10,11, and to forecast and monitor criminal ­behaviour12,13. The implications of these developments are considerable and need to be examined closely. In fact, public authorities are sole providers of service, particularly in
sensitive areas of citizens’ lives, they have privileged access to personal data, take binding decisions, and might
thus inescapably and directly affect beneficiaries and society as a whole.
The magnitude of the problem is determined by the fact that the growing insertion of AI into public service
provision has ushered forms of AI-Governance, i.e., highly complex and dynamic socio-technical configurations
that comprise technological, institutional, and regulatory elements. Specifically, they consist of:
– ADM applications developed for public authorities;
– Institutional practices that define the purposes and forms of usages; and
– A regulatory framework that both enables and constrains ADM deployment.
Algorithms introduce a strong transformative element into these configurations, because they constitute
an inscrutable, opaque system whose operations, choices, and consequences are still poorly understood. They
might encourage or facilitate improper or malicious use by individuals and governments, e.g., for criminal or
1
Department of Social Sciences, Universität Siegen, 57068 Siegen, Germany. 2Université de Lausanne, Institute of
Political Studies, CH‑1015 Lausanne, Switzerland. 3Information Science and Media Studies, Universitetet I Bergen,
5007 Bergen, Norway. 4These authors contributed equally: Christian Lahusen, Martino Maggetti and Marija
Slavkovik. *email: marija.slavkovik@uib.no
Scientific Reports |
(2024) 14:20752
| https://doi.org/10.1038/s41598-024-71761-0
1
Vol.:(0123456789)
www.nature.com/scientificreports/
censoring purposes. Furthermore, they may yield unpredictably inaccurate or biased results, as shown by the
case of Dutch tax authorities who used a self-learning algorithm for spotting child care benefits fraud, which
proved to be biased against lower incomes and ethnic minorities, and violated privacy l­ aws1.
The question then is how aligned is AI-governance with the values of the society that it serves and that
sustains it—particularly core democratic values? In this regard, trust and trustworthiness plays a particularly
important role, because trust is at the same time a precondition, a product, and a foundational ethical value for
functioning human societies. Our claim is that work on AI-governance “trust-alignment” is lacking a profound
multi-disciplinary understanding of how trust and trustworthiness should be conceived and they operate in a
socio-technical context where the main agent of trust and the main bearer of trustworthiness is not human but
an algorithm.
In this paper, we propose to focus on AI-Governance as the main object of inquiry. We argue that the question
of trust and trustworthiness can only be properly addressed, when considering the interplay between its three
components: AI applications, administrative practices, and regulatory systems. Such a research object calls for
an interdisciplinary research agenda that transcends the limited insights of previous studies. We argue that wide
gaps exist in-between the different research fields when it comes to trustworthy AI. Only an interdisciplinary
collaboration, involving sociology, political science, and artificial intelligence, is thus able to generate the necessary synergies to address the challenges associated with the development of value-aligned AI-Governance. The
aim of this paper is thus to provide a selective overview of the core literature in research fields involved in the
analysis of AI-Governance. It wishes to engage computational science, sociology and political science into an
interdisciplinary dialogue aimed at merging available evidence and synchronising research agendas. For this
purpose, the paper pursues the following goals: to (1) justify the relevance of a study of trust and trustworthiness of AI-Governance; (2) provide an overview of the state of the art in the three research fields; and (3) discuss
implications for future research.
Our literature review is a systematic effort to merge interrelated disciplinary research fields. We started by
identifying and defining the trustworthiness of AI-governance as the core research question, and then we engaged
in a comprehensive bibliographic search, article screening and synthesis of findings for the three components of
the socio-technical s­ ystem14. The literature review is wide-ranging, but not exhaustive, because we are dealing
with early accumulation of knowledge in highly dynamic research field, where trust-related issues are increasingly updated. Moreover, relevant evidence is unequally developed, with more consistent efforts in the fields of
the social sciences, when compared to computational science. The aim of our overview is to provide a pluralistic
account of the main issues at stake. Our contribution thus is to build a “point of departure” for scientists that
will engage with the trust and trustworthiness of AI-government.
Trust and trustworthiness
Finding the adequate definition of trust is an arduous undertaking, when considering the highly diverse interdisciplinary field relevant for the study of AI governance. A starting point for such a conceptualisation could
be the proposal to understand trust as “the willingness of a party to be vulnerable to the actions of another
party”15, be this other party another human, an institution or a machine. However, the agreement would only
subsist under the condition that trust is related to human trustors, that is, when conceiving trust as a human
property. Conceptual junctures emerge when considering that trust in humans, institutions and machines are
categorically distinct. Additionally, it is necessary to remember that conceptual disagreements also run across
disciplines, thus contributing to interdisciplinary discrepancies and disagreements. In fact, scholars diverge
conceptually when relating trust to either behaviors, attitudes or ­relations16–19. Anchoring trust in behavioral
practices and norms, in perceptions and preferences and/or in functional or formal properties of interactions
between people, institution and/or machines has considerable implications for the way how trust is conceptualized, operationalized, and analyzed.
An interdisciplinary approach to the study of trust in AI governance should thus abstain from an overhasty
synthesis of available evidence. This is also true for the focus of this collection, namely AI alignment to human
and societal values. Trust is in itself a value, when defining a social value as an ethical conception of what is a
desirable good within a given society. Scholars in various research fields would converge in this assessment, as
trust is widely considered to be an important precondition and ingredient of societal cohesion, political stability,
economic development, and/or technological ­innovation20–24.
However, societal values are intrinsically plural, dynamic and context-dependent, which means that even trust
is a conditional posture. “We learn that, tentatively and conditionally, we can trust trust and distrust distrust,
that it can be rewarding to behave as if we trusted even in unpromising situations”25. Unconditional trust can
transcend what we conceive to be socially desirable, while conditional distrust might be a value in itself—when
voiced through appropriate channels—because it may serve to prevent misbehaviour and keep social relations
or societal systems in c­ heck26–28. Consequently, finding a careful balance between trust and distrust is crucial to
ensure that trust is placed in those who deserve trust, whilst avoiding to place blind trust in the untrustworthy.
The alignment of AI to societal values has thus to consider the pluralism, dynamism and situatedness of
values. This is particularly true with respect to the use of AI systems by public authorities, which are still opaque
and largely unaccountable but potentially deploy effects erga omnes. In this regard, the number of relevant values increases, when considering the specificities of humans, institutions and machines as ‘trustworthy’ trustees.
Scholars in the various research fields have engaged identifying those values that determine the attribution of
trustworthiness, and the list of values that are considered to be relevant exhibit both, similarities and dissimilarities. In regard to interpersonal trust, for instance, it has been highlighted that competence, predictability,
benevolence and integrity are crucial values that help to ascertain a trustee’s ­trustworthiness29,30. With respect
to public institutions the list includes values pertinent for the qualities (e.g., competence, reliability, democratic
Scientific Reports |
Vol:.(1234567890)
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
2
www.nature.com/scientificreports/
participation), procedures (e.g., transparency, accountability) and results (e.g., effectiveness, general welfare,
justice) of political-administrative w
­ ork31. In regard to AI systems, a number of properties have been identified
as trust-relevant, such as reliability, robustness, safety, interpretability, explainability, fairness, transparency, and
accountability. The existence of partially overlapping but also varying lists and typologies of values demonstrates
that AI governance as a socio-technical system has a series of distinct, yet interlocked problems of trustworthiness: there are doubts about the trustworthiness of processes and outputs of artificial intelligence, about the trustworthiness of humans and public authorities to properly handle this technology, and about the trustworthiness
of governments to properly regulate this area of innovation. These problems of trustworthiness are, however,
not necessarily identical, possibly complementary but maybe even incompatible. And this means that value
alignment cannot rely on a simple list of shared values, but requires a careful analysis that helps to ascertain the
specifics and interfaces between the various components of AI governance and their problems of trustworthiness.
Within computational science, the trustworthiness of AI is taken as a long-term goal and under that umbrella,
in recognition of the complexity of the problem, several distinguished networks have been established. This,
however, does not mean that computer scientists have explored the concept of trust and use this to guide the
qualification and quantification of computational and interaction properties associated with trust and trustworthiness of AI. Topics studied with respect to trust and trustworthiness in the AI field pertain to issues as diverse
as reliability, robustness, safety, security, bias, interpretability, privacy, explainability, fairness, transparency,
resilience, and a­ ccountability32,33. For computer scientists it can be very hard to find a good point from which
to learn about and engage with the conceptualisation of trust as studied in the social sciences, as this research is
typically not part of Trustworthy AI agendas in computer science.
Algorithmic decision‑making (ADM), why trust matters
“Algorithmic decision-making” (ADM) is defined by the European parliamentary research s­ ervice34 as computational systems that rely on the analysis of large amounts of personal data, usually using deep learning to infer
correlations or, more generally, to derive information deemed useful to making decisions.
Deep learning re-energised neural network research in the early 2000s due to some fundamental research
breakthroughs. Since then, Neural Networks (NN) have proved to be a useful tool for modelling complex phenomena. In the past decade, these networks have been at the core of widely diffused AI used for image recognition, natural language processing and, most recently, the generation of c­ ontent35.
The ADM as it typically happens is illustrated in Fig. 1, where the decision instance is exemplified by two
human figures, emphasizing that decisions are made about the people directly affected by them. The ADM system
may be used to aid a human in making decisions, as shown by arrow b, by offering a comprehensive analysis of a
large volume of data. Alternatively, the human level of intervention in the decision-making process may vary, as
shown by arrow a, not being made by institution representatives but directly supplied to beneficiaries by the ADM
system. Figure 1 uses a pink brain to illustrate human intervention in- or on-the-loop36,37 of the decision-making.
The European union general data protection regulation (GDPR)38 uses the term “automated decision-making”
to refer to decisions being made without human involvement. Sometimes in the literature, both algorithmic and
automated decision making are used as synonyms. For clarity, we should also state that we define an algorithm
as “an unambiguous procedure to solve a problem or a class of problems. It is typically composed of a set of
instructions or rules that take some input data and return outputs”34.
What separates NN artifacts from previous AI applications is their inbuilt incorrectness of output. Automated decisions that are obtained by utilising NN models can be assessed for accuracy, but unlike other non-AI
computation, we do not have a mathematical guarantee of correctness. This is one of the main reasons why the
intrinsic (or endogenous) trustworthiness of AI becomes such a dominant concern with the rising use of AI.
Notwithstanding these limitations, the automation of tasks in the public sector using algorithmic decisionmaking is relatively widespread in public sector o
­ rganisations39–42.
Fig. 1. An algorithmic decision-making/-aiding system.
Scientific Reports |
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
3
Vol.:(0123456789)
www.nature.com/scientificreports/
The use of ADM increasingly shapes core government functions, namely public service delivery, internal
management, and policy decision making. In this regard, a number of challenges are often mentioned, including
the variance in acceptance, privacy concerns, sample biases, discrimination, surveillance or the digital d
­ ivide43–47.
What is more, ADM implementation is complex and largely dependent on institutional practices and contextual
­factors5. These topics are considered to be urgent, because the automation of routines and decisions by public
authorities deeply affect individuals, for instance, when they apply for social benefit, submit tax declarations,
seek for health services or have violated the law. When a decision is made with the use of AI, public authorities
insert the AI in the middle of the relationship between citizens and state institutions, making its (endogenous and
exogenous) trustworthiness crucial. Due to accelerating technological developments, the trust relationship that
holds between the institution and the individual needs to be re-built when including the AI system and its usage.
An overview of individual disciplines
The insertion of ADM into the relationship between citizens and public authorities has aroused considerable
research ­efforts48–50. Elements of these are devoted to trustworthiness issues and aim to settle a number of
questions: Can AI become trustworthy and how? Can and should citizens and authorities trust a non-human
agent and its usage by humans? Which forms of use by citizens and authorities are adequate? Which regulatory
policies are in place, and which regulatory approach could be established by public authorities to fruitfully mediate adequate trust relationships unfolding between developers, adopters in the public sectors, and citizens as
beneficiaries, with AI/ADM systems in the middle? The three disciplines, computer science, sociology, political
science, have contributed considerably to these debates by providing evidence about the three pillars on which
the topic resides (AI, society, political governance).
In the following sections we give an overview of the trustworthy AI literature in these three fields.
Overview: citizens
The nexus between AI and society has become an increasingly important topic of analysis in the social sciences.
Generally speaking, research has abandoned assumptions of binary relationships between distinct spheres (technology and society), acknowledging that AI is inherently embedded in social r­ eality51,52. Scholars thus tend to
speak of complex socio-technological systems that mirror established structures of social reality: machine-learning systems use informational and behavioural data generated by individual, corporate and state actors, who very
often use decision-making models to replicate standard probabilistic reasoning widely used by humans, and the
impact of machine-learning systems on society depends on the way ADM is used by humans or o
­ rganisations53.
Social sciences have thus painted a nuanced picture of the relation of AI and society, arguing that ADM is transforming but also reproducing societal r­ eality47.
The topics of investigation are as diverse as the implications of artificial intelligence for social reality (e.g., work
and leisure, news coverage and media usage, welfare services and public safety). But in regard to AI-governance,
three topics are of paramount importance.
First, research has been interested in the perceptions and attitudes of citizens in regard to ­AI48,54,55. In spite
of the categorical difference in trusting humans and machines, findings show an increasing readiness within
society to assess AI in regard to trustworthiness, thus mirroring the growing relevance of AI-based applications
in individual’s lives. Not only does trust vary depending on the respondents’ characteristics (e.g., age, gender,
competence, education, personality traits, political orientations), but also regarding institutional and contextual
factors (e.g., the countries’ socio-economic and cultural background)55–59. AI systems and the AI’s perceived
characteristics (e.g., performance, reliability, anthropomorphism, rule-based operation) are crucial, as w
­ ell48,54.
Research thus suggests that citizens are moderately and conditionally trustful, which is considered positive,
because blind trust in untrustworthy socio-technological systems is, in itself, highly ­problematic60.
A second strand of research centres more explicitly on the effects AI systems may have on the readiness of
citizens to trust political institutions. Earlier studies of public trust in e-Government have already shown that
the use of ICT and digitalisation can increase a citizen’s perception of institutional trustworthiness. However,
trust remains highly conditional and depends on the general propensity to trust, the perception of the quality
and usefulness of services, and risk sensitivity, particularly in regard to data privacy and ­security61,62. Digitalisation may also increase public perception of institutional efficiency and transparency, but not necessarily the
recognition of institutional fairness, procedural or distributional ­justice63. The implementation of AI systems
seems to exacerbate these conditionalities, because ADM systems are being adopted primarily for efficiency and
effectivity reasons. At the same time, however, trust in the use of ADM by public authorities greatly depends on
the specific areas of ­application64and the way they are implemented and u
­ sed65.
A third research debate corroborates that conditionality of trust is an adequate public response to the application of ADM systems. ADM systems and their usage have been shown to generate problematic societal consequences, primarily by reproducing, even enlarging, existing social inequalities and power ­asymmetries47,53,66.
This is particularly the case when ADM is used in contexts of public services, as has been shown in regard to
algorithmic profiling in the context of public employment ­services67. Against this backdrop, issues of fairness
and justice have received particular attention by social scientists when assessing A
­ DM68. Before this backdrop,
they (social scientists) were asked whether public acceptability of ADM systems might be influenced by fairness
perceptions. Findings paint an inconclusive picture, as automated decision-making is evaluated on a par with
human decision-making, sometimes even ­better43,67,69,70. Problems are associated with ADM design, but also
with the institutional context of its applications, thus furthering discomfort, and in part, also c­ ontestation71.
Scientific Reports |
Vol:.(1234567890)
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
4
www.nature.com/scientificreports/
Overview: governance
In recent years, concerns about regulation and governance by and of algorithms have emerged as a major challenge for both scholars and policy makers. Public authorities—especially in, but not limited to, high-income
countries-increasingly rely on AI and specifically on ADM systems to inform, assist, and make decisions; in that
regard, for instance, OECD reports have documented the remarkable extent to which countries are integrating AI
into public a­ dministration72. At the same time, governments also face urgent calls to regulate these s­ ystems73–76.
Increasing worries are indeed propagated by many social and political actors about a wide range of unwanted
impacts of AI, and particularly of ADM, that threaten to erode essential collective goods, including democracy,
solidarity, justice, privacy, and individual ­freedom77,78. At the same time, great expectations are formulated about
the beneficial “disruptive” innovation potential of trustworthy ADM in countless areas, ranging from health to
transportation and ­finance45 ­(see79 for a critique). In this sense, a trade-off may emerge between the efficiency
gains associated with the deployment of AI and room for manoeuvre of public authorities to create and implement rules to protect the public ­good80. In that respect, key issues identified in the literature involve the need to
ensure AI accountability by choosing transparent, interpretable models over black-box a­ lternatives81,82 and the
focus on governing the use, instead of the technology itself by discouraging and punishing the abuse of A
­ I83.
It has also been noted that unless specifically regulated, de facto rules by the private ­sector84 will govern the
development and use of AI. Public regulation is nonetheless essential to ensure control over AI and, specifically,
ADM systems and secure their ­trustworthiness82,85–87.
Different regulatory approaches are being developed, adopted, and tentatively implemented, all of which
center on public t­ rust88,89. The EU, in the wake of the AI Act, relies on comprehensive framework inspired by
the precautionary principle, which imposes strict requirements on high-risk applications. The GDPR also has
significant implications for AI, especially concerning data privacy and protection, aiming at ensuring that AI
systems comply with stringent data handling and user consent requirements. For instance, in digital health, this
does not only safeguards patient privacy but also builds public confidence in the use of AI for health purposes.
The US approach, following the Blueprint for an AI Bill of Rights, is more fragmented and sector-specific, with
different agencies overseeing AI use in various areas, while maintaining a conducive environment for innovation
and technological advancement. The UK is developing a pro-innovation regulatory framework, whose goal is to
position the country as a global AI leader, by relying on adaptive and principles-based regulation.
However, it is crucial to recognise that the current political and societal discourses on building trust in AI
often fail to recognise that trust in AI is not unconditionally beneficial, and that focusing exclusively on how to
build public trust in AI systems may obscure the questions about what might be required to actually build trustworthy ­systems60, namely because, as we have seen above, AI systems intrinsically operate in a state of limited
trustworthiness. Rather than blind trust, a certain measure of functional distrust may indeed help to secure the
long-term trustworthiness of AI and ADM systems, which bears the highest potential for transformative change,
but also the highest risk of intrusiveness into people’s private life, especially when it is deployed by public authorities themselves. In this case, the regulation of algorithmic decision-making systems is particularly challenging,
as public authorities take both the role of rule-maker and regulatory t­ arget90.
Given that trust is a relational property, the trustworthiness of AI depends not only on AI itself, but on its
complex interplay with the main actors involved in the operation of AI systems, namely: developers, public
authorities that adopt ADM systems, and citizens/users as beneficiaries. These complex interactions involve
further dilemmas. Developers in the IT sector program, maintain and implement AI systems, and at the same
time, their work can be both facilitated and challenged by AI. Public authorities (including regulators and
intermediaries) face calls to regulate AI while already heavily relying on algorithmic systems to assist decisionmaking, and being confronted with a regulatory issue that is global in s­ cale91,92. Citizens can be empowered by
AI, and benefit from countless new services and opportunities, but biases, errors, new cleavages, social exclusion,
surveillance and privacy protection concerns also loom large.
Overview: computer science
The question that computer science is concerned with when it comes to trust and trustworthiness of AI in ADM
specifically, is which technical tools and properties are relevant or should be developed in this context to make
systems “endogenously” (or intrinsically) trustworthy.
What trusting a non-human agent means has been explored within the field of human-computer and humanrobot interaction, see for e.g.93,94. People appear to have different expectations from machines compared to their
expectations directed to other people, regardless of how intelligent those machines appear to b
­ e93. The trustworthiness that is attached to a non-human agent has a multidimensional structure that involves being capable,
ethical, sincere, and r­ eliable32,95.
Trust and trustworthiness have been extensively used in relation to Artificial Intelligence to signal the need for
adequately embedding AI in a socio-technical s­ ociety96. Chatila et al.32 ground trustworthiness of AI in general
into delivery of service that can be justifiably trusted, and define it to be one that entails the following properties: availability, reliability, safety, confidentiality, integrity, maintainability, and security. Within each of these
sub-topics, thousands of research papers with technical contributions are produced each year and published at
computer science venues. However, although there is much research work under the keywords of trustworthy
AI, not all of it discusses the ­subject33, making it difficult to ascertain where researchers are making progress.
Transparency is seen as quintessential to ­trust97,98, but at the same time, it is also recognised that transparency
is not a “silver bullet”2,99 nor is it clear how it can be operationalised into more trustworthy AI. In the context of
algorithmic decision-making, explainability, in particular, is a very powerful tool in facilitating trust in the process that involves A
­ I100. The field of explainable artificial intelligence (XAI) is sometimes thought to subsume the
work on ­transparency59,101 and sometimes to be a tool for ­transparency98. The field of explainable AI, specifically
Scientific Reports |
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
5
Vol.:(0123456789)
www.nature.com/scientificreports/
the moniker XAI, originates in a 2017 DARPA i­nitiative102, with some very influential papers l­ike73 beginning
to appear in 2016. The field is thus very recent, but also very active. There are numerous systematic analyses
­already59,103 and even a systematic analysis of systematic a­ nalyses104 Work in XAI can range from developing
technical tools ­like105 to meta discussions on what explainability should focus on, for ­example106.
Algorithmic decision-making is of particular concern to the field of algorithmic fairness, which specifically
focuses on fairness in decisions made using machine learning. Algorithmic fairness is as new and as fruitful as
­X AI4,107–109. It studies problems of assessing the societal and individual impact of decisions made or aided by
machine learning algorithms with the aim of ensuring that individuals (individual fairness) or protected groups
(group fairness) are not discriminated against. Beyond assessment, algorithmic fairness is also concerned with
developing methods to remove representational bias from data sets used to train prediction models, and methods
that correct for allocational bias directly in the decision-making algorithms.
Necessarily, algorithmic fairness also studies possible trade-offs between a more efficient, more accurate and
more “fair” decision algorithm. GDPR article 5­ 38 requires that ‘personal data shall be: 1. processed lawfully, fairly
and in a transparent manner in relation to the data subject (‘lawfulness, fairness and transparency’)”. Understandably, it would be practical to have a computational tool that automatically assesses the fairness of an ADM and the
data used for it and by it. There are, at present, numerous proposed mathematical specifications of ­fairness108,109
that are not always grounded in the corresponding ethical d
­ esiderata101,110. Article 21 of the G
­ DPR38 stipulates
the right to object to processing of personal data, and the right to appeal a decision. The possibility for objection
should be afforded by the computational design of the ­ADM111–113, and it is our task to create that affordance.
Contestable AI is a very new field that studies how to make AI systems open and responsive to human intervention throughout their lifecycle, not only after an automated decision has been m
­ ade114.
Discussion
Dealing with the complexities of trust alignment for AI-governance requires joint efforts. On the one hand, the
issue of trust is of paramount relevance with respect to the quick development of AI systems. AI-Governance is
affected by substantial changes located at the level of technological advances, administrative usages and regulatory
policies, whose proper functioning requires the presence of trust among the various actors involved, specifically
including developers, administrative users, or regulators. Public administrators and regulators need to place trust
in the ability of developers to generate trustworthy AI systems that are appropriate for the foreseen administrative usages and comply with regulatory goals and instruments. Similarly, developers need to trustfully rely on
public authorities and regulators to properly employ, monitor and steer the technology in order to capitalize on
its potentials while limiting downsides. For this purpose, it is important to develop sound scientific evidence to
identify the necessary properties of AI systems, administrative usages, and regulations that sustain and enhance
mutual trust, and to demonstrate how the features of trustworthy AI, usages and regulators are deeply intertwined. At the same time, interdisciplinary efforts are needed to determine the limits of trust and the conditions
under which a watchful attitude could emerge and be maintained, as well as the mechanisms allowing for this
vigilance to deploy. It is indeed only through the combination between trust and a watchful attitude (watchful
trust), which implies a certain level of functional distrust, rather than blind trust, that developers, administrative
users, and regulators can anticipate and tame potential risks and harms emanating from technological developments, administrative practices and regulatory measures. In particular, we identify four main challenges that
require future interdisciplinary research efforts.
First, the mushrooming research efforts have contributed to a significant fragmentation of insights. There is
an inflationary use of the concept of trust and t­ rustworthiness33, and little theoretical clarity in regard to trust
in AI, institutions and humans. This is complemented by a plethora of empirical measurements of trust in AI,
institutions and humans, which makes almost impossible to systematically compare the evidence and produce
cumulative knowledge.
Second, studies have provided insights into the acceptability of AI in general, and about the usage of ADM
in institutional contexts. However, research is dominated by single case studies and lab experiments that say
little about long-term dynamics and real-life settings, and are blind with regard to situational and contextual
conditions. We thus know very little about whether and how ADM systems and practices contribute to the
formation of trust and distrust. Additionally, the dominance of single case or country-specific analyses have led
to inconclusive findings.
Thirdly, notwithstanding early attempts, which are fruitful but mainly set out to explore and scope the fi
­ eld83,92,
research has yet to systematically develop an explicit research agenda addressing the regulatory challenges and
responses to the technological transformations spurred by ADM systems and their use.
Fourth, research has not yet provided a consistent answer to the question of whether ADM systems and their
utilisation can and should be trustworthy. There is an implicit agreement that blind trust is, generally speaking,
inadequate posture in complex technological, social and political contexts, inasmuch as disconsolate distrust
will be harmful.
In conclusion, to make substantial progress on the study AI value alignment, we need to first understand
values, not only from the perspective of humans and human society, but also—and above all—from the perspective of machines, and how such values are intertwined and possibly interact.
Conclusion
Considerable progress has been made in recent years in deepening our knowledge about trustworthy AI governance. This is due to the increasing societal and scientific relevance of AI, the growing need to provide robust
knowledge, and intensifying research efforts in a number of research fields. However, we still see limits in our
understanding of trust and the trustworthiness of AI governance.
Scientific Reports |
Vol:.(1234567890)
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
6
www.nature.com/scientificreports/
We identify four challenges that require future interdisciplinary research efforts. “Interdisciplinary” is always
desirable but not trivial to achieve. To overcome the fragmentation of insights (first challenge) we need to lower
the barrier to understanding across researchers who study citizens, governance and AI on one end and those
who “create” AI, on the other end. This may be accomplished by providing a simplified “interface” to a specific
discipline: identifying the critical ideas, research programs, and accomplishments in each of the concerned
research areas. This article is intended to be a “prototype” of such “interface” work.
A successfully discipline interfacing will enable empirical and theoretical framework of analysis that helps
to identify adequate forms of trustworthy ADM systems, societal usages, and political regulations grounded on
the concept of watchful trust. This work needs to be aligned with a systematic, comparative research agenda that
empirically examines how the relevance of the trust and trustworthiness properties may vary, and how crosssectional, longitudinal, and, respectively, contextual factors shape and condition different levels and forms of
trust. Lastly, regulatory regimes are currently being developed from the perspective of regulation and governance of AI. This development needs to be aligned with and investigation of how they address the problem of AI
trustworthiness (or fail to do so), and, in turn, how these embryonic regimes reshape trust and distrust in AI
and in ADM.
In conclusion, to make substantial progress on the study of AI value alignment, we need to first understand
values, not only from the perspective of humans and human society, but also—and above all—from the perspective of machines, and how such values are intertwined and possibly interact. Values that unequivocally require
understanding from the perspective of machines are trust and trustworthiness. Progress in AI Governance
depends on this understanding.
Data availability
This study does not use datasets. The articles datasets used and/or analyzed during the current study are available
from the corresponding author upon reasonable request.
Received: 18 March 2024; Accepted: 30 August 2024
References
1. Heikkilä, M. & Heikkilä, M. Dutch scandal serves as a warning for Europe over risks of using algorithms. POLITICO (2022).
https://​www.​polit​ico.​eu/​artic​le/​dutch-​scand​al-​serves-​as-a-​warni​ng-​for-​europe-​over-​risks-​of-​using-​algor​ithms/.
2. Knowles, B., Fledderjohann, J., Richards, J. T. & Varshney, K. R. Trustworthy ai and the logics of intersectional resistance. In
Proc. 2023 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’23, 172–182, (Association for Computing
Machinery, USA, 2023).
3. Jiang, Y. et al. Quo vadis artificial intelligence?. Discov. Artif. Intell. https://​doi.​org/​10.​1007/​s44163-​022-​00022-8 (2022).
4. Kearns, M. & Roth, A. The Ethical Algorithm: The Science of Socially Aware Algorithm Design (Oxford University Press, 2019).
5. Neumann, O., Guirguis, K. & Steiner, R. Exploring artificial intelligence adoption in public organizations: A comparative case
study. Pub. Manag. Rev. 26, 114–141 (2023).
6. Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A. & Vaithianathan, R. Toward Algorithmic Accountability in
Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare
Services. In Proc. 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, 1–12, (Association for Computing
Machinery, New York, NY, USA, 2019).
7. Chouldechova, A., Benavides-Prado, D., Fialko, O. & Vaithianathan, R. A case study of algorithm-assisted decision making in
child maltreatment hotline screening decisions. In Proc. 1st Conference on Fairness, Accountability and Transparency, vol. 81,
134–148 (Proceedings of Machine Learning Research, 2018).
8. Flügge, A. A. Perspectives from practice: Algorithmic decision-making in public employment services. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing, CSCW ’21, 253–255, https://​doi.​
org/​10.​1145/​34622​04.​34817​87 (Association for Computing Machinery, New York, NY, USA, 2021).
9. Sarlin, R. Automationin administrative decision-makingconcerningsocialbenefits: A government agency perspective. In The
Rule of Law and Automated Decision-Making (ed. Suksi, M.) (Springer, 2023).
10. Asquith, R. Tax authorities adopt AI for tax fraud and efficiencies-vatcalc.com. Section: Artificial Intelligence (2023).
11. de la Feria, R. & Grau Ruiz, M. A. The Robotisation of tax administration. In Interactive Robotics: Legal, Ethical, Social and
Economic Aspects (ed. Grau, A.) (Springer Nature, 2022).
12. Mugari, I. & Obioha, E. E. Predictive policing and crime control in the United States of America and Europe: Trends in a decade
of research and the future of predictive policing. Soc. Sci. 10, 234. https://​doi.​org/​10.​3390/​socsc​i1006​0234 (2021).
13. Van Brakel, R. How to watch the watchers? Democratic oversight of algorithmic police surveillance in Belgium. Surveill. Soc.
19, 228–240. https://​doi.​org/​10.​24908/​ss.​v19i2.​14325 (2021).
14. Nowell, L. et al. Interdisciplinary mixed methods systematic reviews: Reflections on methodological best practices, theoretical
considerations, and practical implications across disciplines. Soc. Sci. Humanit. Open 6, 100295. https://​doi.​org/​10.​1016/j.​ssaho.​
2022.​100295 (2022).
15. Mayer, R. C., Davis, J. H. & Schoorman, F. D. An integrative model of organizational trust. Acad. Manag. Rev. 20, 709–734 (1995).
16. Schilke, O., Reimann, M. & Cook, K. S. Trust in social relations. Annu. Rev. Sociol. 47, 239–259. https://​doi.​org/​10.​1146/​annur​
ev-​soc-​082120-​082850 (2021).
17. Zmerli, S. Political trust. In Encyclopedia of Quality of Life and Well-Being Research (ed. Maggino, F.) (Springer International
Publishing, 2020).
18. Nguyen, C. T. Trust as an unquestioning attitude. In Oxford Studies in Epistemology (ed. Nguyen, C. T.) (Oxford University Press,
2022).
19. Luhmann, N. Trust and Power (Wiley, 1979).
20. Rodriguez-Soto, M., Serramia, M., Lopez-Sanchez, M. & Rodriguez-Aguilar, J. A. Instilling moral value alignment by means of
multi-objective reinforcement learning. Eth. Inf. Technol. 24, 9. https://​doi.​org/​10.​1007/​s10676-​022-​09635-0 (2022).
21. Arnold, T., Kasenberg, D. & Scheutz, M. Value alignment or misalignment - what will keep systems accountable? In The Workshops of the The Thirty-First AAAI Conference on Artificial Intelligence, Saturday, February 4–9, 2017, San Francisco, California,
USA, vol. WS-17 of AAAI Technical Report (AAAI Press, 2017).
22. Gabriel, I. Artificial intelligence, values, and alignment. Minds Mach. 30, 411–437. https://​doi.​org/​10.​1007/​s11023-​020-​09539-2
(2020).
Scientific Reports |
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
7
Vol.:(0123456789)
www.nature.com/scientificreports/
23. Sutrop, M. Challenges of aligning artificial intelligence with human values. Acta Balt. Hist. Et Philos. Sci. 8, 54–72. https://​doi.​
org/​10.​11590/​abhps.​2020.2.​04 (2020).
24. Hirschman, A. O. Against parsimony: Three easy ways of complicating some categories of economic. Am. Econ. Rev. 74, 89–96.
https://​doi.​org/​10.​1017/​S0266​26710​00018​63 (1984).
25. Gambetta, D. Can we trust trust? In Making and Breaking Cooperative Relations (ed. Gambetta, D.) (Basil Blackwell, 1988).
26. Lewicki, R. J., McAllister, D. & Bies, R. J. Trust and distrust: New relationships and realities. Acad. Manag. Rev. 23, 438–458
(1998).
27. Sztompka, P. Trust distrust and two paradoxes of democracy. Eur. J. Soc. Theory 1, 19–32. https://​doi.​org/​10.​1177/​13684​31980​
01001​003 (1998).
28. Warren, M. Trust and democracy. In The Oxford Handbook on Social and Political Trust (ed. Uslaner, E.) (Oxford University
Press, 2018).
29. Bacharach, M. & Gambetta, D. Trust in signs. In Trust in Society (ed. Cook, K. S.) (Russell Sage Foundation, 2001).
30. Lewicka, D. & Zakrzewska-Bielawska, A. F. Trust and distrust in interorganisational relations—Scale development. PLoS ONE
https://​doi.​org/​10.​1371/​journ​al.​pone.​02792​31 (2022).
31. Levi, M. & Stoker, L. Political trust and trustworthiness. Annu. Rev. Polit. Sci. 3, 475–507. https://​doi.​org/​10.​1146/​annur​ev.​polis​
ci.3.​1.​475 (2000).
32. Chatila, R. et al. Trustworthy ai. In Reflections on Artificial Intelligence for Humanity (eds Braunschweig, B. & Ghallab, M.)
(Springer International Publishing, 2021).
33. Probasco, E. S., Toney, A. S. & Curlee, K. T. The Inigo Montoya problem for trustworthy AI. The use of keywords in policy and
research. Technical Report Center Security and Emerging Technologies. (2023). https://​cset.​georg​etown.​edu/​publi​cation/​the-​
inigo-​monto​ya-​probl​em-​for-​trust​worthy-​ai/.
34. Castelluccia, C. & Le Métayer, D. Understanding algorithmic decision-making: Opportunities and challenges https://​www.​europ​
arl.​europa.​eu/​think​tank/​en/​docum​ent/​EPRS_​STU(2019)​624261 (European Union, Brussels, 2019).
35. Bengio, Y., Lecun, Y. & Hinton, G. Deep learning for AI. Commun. ACM 64, 58–65. https://​doi.​org/​10.​1145/​34482​50 (2021).
36. Christopher, S., Calhoun, J. J. G., Philip, B. & Lyons, J. B. Linking precursors of interpersonal trust to human-automation trust:
An expanded typology and exploratory experiment. J. Trust Res. 9, 28–46. https://​doi.​org/​10.​1080/​21515​581.​2019.​15797​30
(2019).
37. Fischer, J. E. et al. In-the-loop or on-the-loop? Interactional arrangements to support team coordination with a planning agent.
Concurr. Comput. Pract. Exp. 33, e4082. https://​doi.​org/​10.​1002/​cpe.​4082 (2021).
38. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons
with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General
Data Protection Regulation) http://​data.​europa.​eu/​eli/​reg/​2016/​679/​oj (2016).
39. Binder, N. B. et al. Einsatz Künstlicher Intelligenz in der Verwaltung: rechtliche und ethische Fragen. https://​www.​zh.​ch/​conte​
nt/d
​ am/z​ hweb/b
​ ilder-d
​ okume​ nte/t​ hemen/p
​ oliti​ k-s​ taat/k​ anton/d
​ igita​ le-v​ erwal​ tung-u
​ nd-e-g​ overn
​ ment/p
​ rojek​ te_d
​ igita​ le_t​ rans​
forma​tion/​ki_​einsa​tz_​in_​der_​verwa​ltung_​2021.​pdf (2021).
40. Loi, M., Mätzener, A., Müller, A. & Spielkamp, M. Automated Decision-Making Systems in the Public Sector: An Impact Assessment Tool for Public Authorities. Tech. Rep., algorithmwatch.org. https://​algor​ithmw​atch.​org/​en/​wp-​conte​nt/​uploa​ds/​2021/​06/​
ADMS-​in-​the-​Public-​Sector-​Impact-​Asses​sment-​Tool-​Algor​ithmW​atch-​June-​2021.​pdf. (2021).
41. de Sousa, W. G., de Melo, E. R. P., Bermejo, P. H. D. S., Farias, R. A. S. & Gomes, A. O. How and where is artificial intelligence
in the public sector going? A literature review and research agenda. Gov. Inf. Q. 36, 101392. https://​doi.​org/​10.​1016/j.​giq.​2019.​
07.​004 (2019).
42. Spielkamp, M. Automating Society: Taking Stock of Automated Decision-Making in the EU. https://​algor​ithmw​atch.​org/​en/​
wp-​conte​nt/​uploa​ds/​2019/​02/​Autom​ating_​Socie​ty_​Report_​2019.​pdf. (2019).
43. Araujo, T. et al. In AI we trust? Perceptions about automated decision-making by artificial intelligence. AI Soc. 35, 611–623.
https://​doi.​org/​10.​1007/​s00146-​019-​00931-w (2020).
44. Fazelpour, S. & Danks, D. Algorithmic bias: Senses, sources, solutions. Philos. Compass. https://​doi.​org/​10.​1111/​phc3.​12760
(2021).
45. van Noordt, C. & Misuraca, G. Artificial intelligence for the public sector: results of landscaping the use of AI in government
across the European Union. Gov. Inf. Q. 39, 101714 (2022).
46. Wirtz, B. W., Weyerer, J. C. & Sturm, B. J. The dark sides of artificial intelligence: An integrated AI governance framework for
public administration. Int. J. Pub. Adm. 43, 818–829 (2020).
47. Zajko, M. Artificial intelligence, algorithms, and social inequality: Sociological contributions to contemporary debates. Soc.
Compass. https://​doi.​org/​10.​1111/​soc4.​12962 (2022).
48. Glikson, E. & Woolley, A. W. Human trust in artificial intelligence: review of empirical research. Acad. Manag. Ann. 14, 627–660.
https://​doi.​org/​10.​5465/​annals.​2018.​0057 (2020).
49. Marcus, G. & Davis, E. Rebooting AI: Building artificial intelligence we can trust (Vintage, 2019).
50. Rossi, F. Building trust in artificial intelligence. J. int. Aff. 72, 127–134 (2018).
51. Lange, A. C., Lenglet, M. & Seyfert, R. On studying algorithms ethnographically: Making sense of objects of ignorance. Organization 26, 598–617. https://​doi.​org/​10.​1177/​13505​08418​808230 (2019).
52. Seyfert, R. Algorithms as regulatory objects. Inform. Commun. Soc. 25, 1542–1558 (2022).
53. Gerdon, F., Bach, R. L., Kern, C. & Kreuter, F. Social impacts of algorithmic decision-making: A research agenda for the social
sciences. Big Data Soc, https://​doi.​org/​10.​1177/​20539​51722​10893​05 (2022).
54. Kaplan, A. D., Kessler, T. T., Brill, J. C. & Hancock, P. A. Trust in artificial intelligence: Meta-analytic findings. Hum. Fact. 65,
337–359. https://​doi.​org/​10.​1177/​00187​20821​10139​88 (2023).
55. Lockey, S., Gillespie, N., Holm, D. & Someh, I. A. A review of trust in artificial intelligence: Challenges, vulnerabilities and future
directions. Proc. 54th Hawaii Int. Conf. on Syst. Sci. https://​doi.​org/​10.​24251/​hicss.​2021.​664 (2021).
56. Chen, Y. N. K. & Wen, C. H. R. Impacts of attitudes toward government and corporations on public trust in artificial intelligence.
Commun. Stud. 72, 115–131 (2021).
57. Choung, H., David, P. & Ross, A. Trust and ethics in AI. AI Soc. 38, 733–745 (2023).
58. Molina, M. D. & Sundar, S. S. Does distrust in humans predict greater trust in AI? Role of individual differences in user responses
to content moderation. New Media Soc. https://​doi.​org/​10.​1177/​14614​44822​11035​34 (2022).
59. Schepman, A. & Rodway, P. The general attitudes towards artificial intelligence scale (gaais): Confirmatory validation and associations with personality, corporate distrust, and general trust. Int. J. Hum. Comput. Interact. 39, 2724–2741. https://​doi.​org/​10.​
1080/​10447​318.​2022.​20854​00 (2023).
60. Krüger, S. & Wilson, C. The problem with trust: On the discursive commodification of trust in ai. AI Soc. 38, 1753–1761 (2023).
61. Colesca, S. E. Understanding trust in e-government. Econ. Eng. Decis. 3, 7–15 (2009).
62. Ejdys, J., Ginevicius, R., Rozsa, Z. & Janoskova, K. The role of perceived risk and security level in building trust in e-government
solutions. E+XXII https://​doi.​org/​10.​15240/​tul/​001/​2019-3-​014 (2019).
63. Smith, M. L. Limitations to building institutional trustworthiness through e-government: A comparative study of two e-services
in Chile. J. Inf. Technol. 26, 78–93. https://​doi.​org/​10.​1057/​jit.​2010.​17 (2011).
64. Aoki, N. An experimental study of public trust in AI chatbots in the public sector. Gov. Inf. Q. 37, 101490 (2020).
Scientific Reports |
Vol:.(1234567890)
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
8
www.nature.com/scientificreports/
65. Kreps, S. & Jakesch, M. Can AI communication tools increase legislative responsiveness and trust in democratic institutions?.
Gov. Inf. Q. 40, 101829 (2023).
66. Maas, J. Machine learning and power relations. AI Soc. 38, 1493–1500. https://​doi.​org/​10.​1007/​s00146-​022-​01400-7 (2023).
67. Kern, C., Bach, R. L., Mautner, H. & Kreuter, F. Fairness in Algorithmic Profiling: A German Case Study. CoRR abs/2108.04134
(2021).
68. Kuppler, M., Kern, C., Bach, R. L. & Kreuter, F. From fair predictions to just decisions? Conceptualizing algorithmic fairness
and distributive justice in the context of data-driven decision-making. Front. Sociol. 7, 883999. https://​doi.​org/​10.​3389/​fsoc.​
2022.​883999 (2022).
69. Helberger, N., Araujo, T. & de Vreese, C. H. Who is the fairest of them all? Public attitudes and expectations regarding automated
decision-making. Comput. Law Secur. Rev. 39, 105456. https://​doi.​org/​10.​1016/j.​clsr.​2020.​105456 (2020).
70. Miller, S. M. & Keiser, L. R. Representative bureaucracy and attitudes toward automated decision making. J. Pub. Adm. Res.
Theory 31, 150–165. https://​doi.​org/​10.​1093/​jopart/​muaa0​19 (2021).
71. Kaun, A. Suing the algorithm: The mundanization of automated decision-making in public services through litigation. Inform.
Commun. Soc. 25, 2046–2062 (2022).
72. Berryhill, J., Heang, K. K., Clogher, R. & McBride, K. Hello World: Artificial Intelligence and its Use in the Public Sector (OECD,
2019).
73. Buiten, M. C. Towards intelligent regulation of artificial intelligence. Eur. J. Risk Regul. 10, 41–59 (2019).
74. Burrell, J. & Fourcade, M. The society of algorithms. Annu. Rev. Sociol. 47, 213–237. https://d
​ oi.o
​ rg/1​ 0.1​ 146/a​ nnure​ v-s​ oc-0​ 90820-​
020800 (2021).
75. Justo-Hanani, R. The politics of artificial Intelligence regulation and governance reform in the European union. Policy Sci. 55,
137–159 (2022).
76. Yeung, K. & Lodge, M. Algorithmic Regulation (Oxford University Press, 2019).
77. Ulbricht, L. & Yeung, K. Algorithmic regulation: A maturing concept for investigating regulation of and through algorithms.
Regul. Gov. 16, 3–22 (2022).
78. Zuboff, S. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (Public Affairs, 2019).
79. Bourne, C. AI cheerleaders: Public relations, neoliberalism and artificial intelligence. Pub. Relat. Inq. 8, 109–125. https://​doi.​
org/​10.​1177/​20461​47X19​835250 (2019).
80. Gritsenko, D. & Wood, M. Algorithmic governance: A modes of governance approach. Regul. Gov. 16, 45–62 (2022).
81. Busuioc, M. Accountable artificial intelligence: Holding algorithms to account. Pub. Adm. Rev. 81, 825–836. https://​doi.​org/​10.​
1111/​puar.​13293 (2021).
82. Grimmelikhuijsen, S. Introduction to the Digital Government and Artificial Intelligence Minitrack. In Proceedings of the 55th
Hawaii International Conference on System Sciences (2022).
83. Büthe, T., Djeffal, C., Lütge, C., Maasen, S. & Ingersleben-Seip, N. V. Governing AI—Attempting to herd cats? Introduction to
the special issue on the governance of artificial intelligence. J. Eur. Pub. Policy 29, 1721–1752. https://​doi.​org/​10.​1080/​13501​763.​
2022.​21265​15 (2022).
84. Nitzberg, M. & Zysman, J. Algorithms, data, and platforms: The diverse challenges of governing AI. J. Eur. Pub. Policy 29,
1753–1778 (2022).
85. Busuioc, M. AI Algorithmic Oversight: New Frontiers in Regulation (Edward Elgar Publishing, 2022).
86. Russell, S. Artificial intelligence and the problem of control. In Perspectives on Digital Humanism (eds Hannes, W. et al.) (Springer,
2022).
87. Six, F. & Verhoest, K. Trust in Regulatory Regimes (Edward Elgar Publishing, 2017).
88. Buiten, M. C. Towards intelligent regulation of artificial intelligence. Eur. J. Risk Regul. 10, 41–59. https://​doi.​org/​10.​1017/​err.​
2019.8 (2019).
89. Justo-Hanani, R. The politics of artificial intelligence regulation and governance reform in the European union. Policy Sci. 55,
137–159. https://​doi.​org/​10.​1007/​s11077-​022-​09452-8 (2022).
90. Di Mascio, F., Maggetti, M. & Natalini, A. Exploring the dynamics of delegation over time: Insights from Italian anti-corruption
agencies (2003–2016). Policy Stud. J. 48, 367–400. https://​doi.​org/​10.​1111/​psj.​12253 (2020).
91. Abbott, K. W., Levi-faur, D. & Snidal, D. Theorizing regulatory intermediaries: The RIT model. Ann. Am. Acad. Polit. Soc. Sci.
670, 14–35. https://​doi.​org/​10.​1177/​00027​16216​688272 (2017).
92. Tallberg, J. et al. The Global Governance of Artificial Intelligence: Next Steps for Empirical and Normative Research. ArXiv:​
2305.​11528 (2023).
93. Hidalgo, C. A., Orghian, D., Albo Canals, J., de Almeida, F. & Martin, N. How Humans Judge Machines (The MIT Press, 2021).
94. Ingram, M. Calibrating trust between humans and artificial intelligence systems. PhD Thesis, University of Glasgow (2023).
95. Ullman, D. & Malle, B. F. What Does it Mean to Trust a Robot? Steps Toward a Multidimensional Measure of Trust. In Companion
of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, 263–264, https://​doi.​org/​10.​1145/​31733​86.​31769​
91 (Association for Computing Machinery, New York, NY, USA, 2018).
96. Ethics guidelines for trustworthy AI|Shaping Europe’s digital future (2019).
97. vonEschenbach, W. Transparency and the black box problem: Why we do not trust AI. Philos. Technol. 34, 1607–1622. https://​
doi.​org/​10.​1007/​s13347-​021-​00477-0 (2021).
98. Winfield, A. F. T. et al. P7001: A proposed standard on transparency. Front. Robot. AI https://​doi.​org/​10.​3389/​frobt (2021).
99. Wang, H. Why should we care about the manipulative power of algorithmic transparency?. Philos. Technol. 36, 9 (2023).
100. Grimmelikhuijsen, S. Explaining why the computer says no: Algorithmic transparency affects the perceived trustworthiness of
automated decision-making. Pub. Adm. Rev. 83, 241–262. https://​doi.​org/​10.​1111/​puar.​13483 (2023).
101. Floridi, L. & Cowls, J. A unified framework of five principles for ai in society. Harv. Data Sci. Rev. https://​doi.​org/​10.​1162/​99608​
f92.​8cd55​0d1 (2019).
102. Turek, M. Explainable Artificial Intelligence (XAI) (2017).
103. Speith, T. A review of taxonomies of explainable artificial intelligence (XAI) methods. In Proc. 2022 ACM Conference on Fairness,
Accountability, and Transparency (FAccT ’22), 2239–2250, https://d
​ oi.o
​ rg/1​ 0.1​ 145/3​ 53114​ 6.3​ 53463​ 9 (Association for Computing
Machinery, New York, NY, USA, 2022).
104. Schwalbe, G. & Finzel, B. A comprehensive taxonomy for explainable artificial intelligence: A systematic survey of surveys on
methods and concepts. Data Min. Knowl. Disc. https://​doi.​org/​10.​1007/​s10618-​022-​00867-8 (2023).
105. Ribeiro, M. T., Singh, S. & Guestrin, C. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proc. 22nd
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, 1135–1144, https://​doi.​org/​10.​
1145/​29396​72.​29397​78 (Association for Computing Machinery, New York, NY, USA, 2016).
106. Miller, T. Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven Decision Support using Evaluative AI. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 333–342, https://​doi.​org/​10.​1145/​35930​13.​
35940​01 (2023).
107. Chouldechova, A. & Roth, A. A snapshot of the frontiers of fairness in machine learning. Commun. ACM 63, 82–89. https://​
doi.​org/​10.​1145/​33768​98 (2020).
108. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. & Galstyan, A. A survey on bias and fairness in machine learning. ACM
Comput. Surv. https://​doi.​org/​10.​1145/​34576​07 (2021).
Scientific Reports |
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
9
Vol.:(0123456789)
www.nature.com/scientificreports/
109. Pessach, D. & Shmueli, E. A review on fairness in machine learning. ACM Comput. Surv. 55, 1–44 (2022).
110. Kasirzadeh, A. Algorithmic Fairness and Structural Injustice: Insights from Feminist Political Philosophy. In Proc. 2022 AAAI/
ACM Conference on AI, Ethics, and Society, 349–356, https://​doi.​org/​10.​1145/​35140​94.​35341​88 (Association for Computing
Machinery, 2022).
111. Almada, M. Human intervention in automated decision-making: Toward the construction of contestable systems. In Proc. 17th
International Conference on Artificial Intelligence and Law, 2–11 (2019).
112. Henin, C. & Le Métayer, D. Beyond explainability: Justifiability and contestability of algorithmic decision systems. AI & Soc.
(2021).
113. Lyons, H., Velloso, E. & Miller, T. Conceptualising contestability: Perspectives on contesting algorithmic decisions. Proc. ACM
Hum. Comput. Interact. 5, 1–25 (2021).
114. Alfrink, K., Keller, I., Kortuem, G. & Doorn, N. Contestable AI by design: Towards a framework. Minds Mach. 33, 613–639
(2022).
Author contributions
All authors contributed equally to this work. State of the art computer science was written by Slavkovik. State of
the art governance was written by Maggetti. State of the art citzens was written by Lahusen.
Funding
Open access funding provided by University of Bergen.
Competing interests
The authors declare no competing interests.
Additional information
Correspondence and requests for materials should be addressed to M.S.
Reprints and permissions information is available at www.nature.com/reprints.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and
indicate if changes were made. The images or other third party material in this article are included in the article’s
Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included
in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or
exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy
of this licence, visit http://creativecommons.org/licenses/by/4.0/.
© The Author(s) 2024
Scientific Reports |
Vol:.(1234567890)
(2024) 14:20752 |
https://doi.org/10.1038/s41598-024-71761-0
10