See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/370766004
Operationalizing Ethics for Information Systems Design-A Tool for Ethical
Software
Conference Paper · August 2023
CITATIONS
READS
0
20
1 author:
Olga Levina
Brandenburg University of Applied Sciences
67 PUBLICATIONS 475 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Research in Social Media View project
Design and Application of Socially-aware information systems View project
All content following this page was uploaded by Olga Levina on 15 May 2023.
The user has requested enhancement of the downloaded file.
Ethical Software Assessment Tool (ESAT)
Operationalizing Ethics for Information
Systems Design- A Tool for Ethical Software
Assessment
Emergent Research Forum (ERF) Paper
Olga Levina
Brandenburg University of Applied
Sciences
levina@th-brandenburg.de
Abstract
Propagation of information technology in the increasingly diverse domains, raises question concerning
moral implications. While the number of ethical principles in the IT design is growing, they do not
necessarily apply to specific products, which means developers and e.g. purchasing agent need to
contextualize them for each specific IT product. This paper describes an actionable artifact that allows
assessment of ethical non-functional requirements of an information system through operationalization
of ethical values, Ethical Software Assessment Tool (ESAT). It is based on a survey of values that are
currently considered in research and practice as well as in legal initiatives in IT context. The ethical
approach and principles operationalized in ESAT are presented and discussed.
Keywords
Information systems development, ethical assessment, usable artifact.
Introduction
Based on process performance metrics (Živadinović et al., 2014) efficiency is often a key measurement for
successful products in Information Systems Development (ISD) (Antolić, 2008). Nevertheless, with the
increasing discussion about machine learning-based systems (MLS), non-functional requirements, related
to ethical issues, such as transparency, explainability, and usability have moved into the focus of business
(Vakkuri et al., 2022), individual (Galaz et al., 2021), and public (Jobin et al., 2019) discussions. Despite
the ongoing debates, companies still rely largely on theory-focused guidelines and do not yet have clear
tools or methods for implementing ethics in their software products (Vakkuri et al., 2022). Moreover,
recent empirical studies (Griffin et al., 2023; Herwix et al., 2022) have shown that software developers do
not consider ethical principles as an aspect that requires closer attention or resources from the
development team. The reasons mentioned are the lack of resources, lack of ethical expertise in the
development team, and lack of technical knowledge on the user side that is needed, e.g., to understand the
explanations included due to the requirement for transparency (Griffin et al., 2023; Vakkuri et al., 2022).
This emerging research paper describes an actionable artifact that allows assessment of ethical nonfunctional requirements of an information system through operationalization of ethical values. Ethical
Software Assessment Tool (ESAT1) can be used to evaluate both already launched software products and
software products under development. Identifying the values that lead to product development can be
useful for assessing and differentiating multiple products with similar functionality, while also allowing
software and product designers to recognize an ethical issue that needs to be resolved, analyzed, or cued
into the next release cycle. Enacting this assessment will also move ethical principles from the discussion
to the operative level, proving users and developer teams with actionable moral agency.
1https://t1p.de/esat_v1
Twenty-ninth Americas Conference on Information Systems, Panama, 2023
1
Ethical Software Assessment Tool (ESAT)
ESAT is a tool realizes as an online survey of the values. These relevance of the surveyed approaches is
described in the following section. The ethical approach and principles operationalized in ESAT are then
presented. A discussion and outlook on the further development of this emerging research completes this
paper.
Ethical approaches and software development
With the discussion about innovative technologies, particularly that involving MLS, also known as
artificial intelligence (AI), the topic of ethics has moved further into the focus of software users and
auditors. Legislation initiatives, e,g. such as the currently discussed AI Act (EIPA, 2023) in the EU,
provide non-functional but legally binding requirements for ISD. Being based on public and philosophical
discussion, these requirements are theory-based and leave developers without realization details (Vakkuri
et al., 2022). Nevertheless, the discussion about ethics in IT is not new. (Mason, 1986) discussed the
ethical issues that would be relevant in a digital world and defined the four main categories: Privacy,
Accessibility, Property and Accuracy. Value Sensitive Design (VSD) is an approach which goal it is to
encourage software designers to consider different stakeholder values in the design process (Friedman, B.,
Kahn, P., Borning, 2002). VSD has gained some attention in the context of information systems research.
Following 13 values were in the context of ISD were suggested (Kahn & Friedman, 2007): Human
Welfare, Ownership and Property, Privacy, Freedom from Bias, Universal Usability, Trust, Autonomy,
Informed Consent, Accountability, Courtesy, Identity, Calmness, and Environmental Sustainability.
Nevertheless, VSD has not yet found a broad acknowledgement in the practice and was criticized for
lacking in pragmatism and methodological guidance (Winkler & Spiekermann, 2021). Principles are one
of the ways of categorizing the discussion in the area. To bridge the gap between theory and practice,
guidelines have been used to derive usable artifacts. Based on an analysis of the numerous AI ethics
guidelines that now exist, Jobin et al. (2019) listed the key principles that could be considered central
based on how often they appear in these guidelines: “transparency, justice and fairness, non-maleficence,
responsibility, privacy, beneficence, freedom and autonomy, trust, dignity, sustainability, and solidarity”.
However, the number of sources, analyses, and comparisons of ethical principles considering the
development of MLS is growing (Mcgrath et al., 2022). Yet, past research has shown that guidelines are
rarely effective in software engineering (Vakkuri & Abrahamsson, 2018). Hence, Harris et al. (2011)
developed DIAODE, a meta-methodology for the ethical assessment of new and emerging technologies. It
was developed with the goal to be helpful in teaching and professional training to show how existing
guidance, such as law, codes of conduct and principles from ethical theory can be used to further ethical
decision-making as suggested by Kallman & Grillo (1996) for software engineers. (Vakkuri et al., 2021)
suggested ECCOLA, a method for implementing AI ethics that is built on various AI ethics guidelines,
providing a more practical support for companies. From a more philosophical perspective, Chatterjee &
Fuller (2009) outline an assessment of traditional methodical approaches to ISD, contending that many
of these approaches' underlying assumptions—such as universality and rationality—lead to insufficient
ontological and epistemological considerations. This pattern frequently leads to failures of information
systems. The authors suggest to combine ethical analysis with conventional ISD methods to solve some of
the drawbacks of traditional ISD. Furthermore, the authors argue that giving more attention to the moral
duty of important ISD participants, such as the team of analysts, may enhance the ISD process. ESAT
builds on the discussions and insights described above and implements them into a usable IT artifact for
users, developers, researchers and procurement managers allowing the inclusion of moral duty into the
ISD.
Ethical Software Assessment Tool (ESAT)
ESAT is designed for the assessment of ethical aspects of a new technology from both, a strategic or
abstract perspective. Guiding questions for the abstract perspective might be: does the information
system consider relevant ethical aspects? Are there any non-functional aspects that need to be included or
ethical issues that need to be discussed (before the product launch or for the new release)? Guiding
questions from the strategic perspective might be: does the information system fit with the contextual
ethical requirements? Are the software features distinguished enough to provide a unique product? Some
of the existing approaches to the ethical analysis, e.g. DIODE by Harris et al. (2011) or Kallman & Grillo
(1996) start with the definition of ethical issues. It is argued here, that to identify an ethical issue, ethical
Twenty-ninth Americas Conference on Information Systems, Panama, 2023
2
Ethical Software Assessment Tool (ESAT)
analysis process needs to have been already started. To do so, ethical expertise is required. However,
ESAT does not suggest that the software product has an existing ethical issue. It is used to assess the
current state of the product along ethical principles discussed in the IT context. The assessment is guided
along operationalized ethical values and therefore does not require any ethical expertise or primary issue
analysis. After the assessment, the auditor (developer, client, project manager or any other role in the ISD
or purchase team) can decide on the following action or whether an action is needed at all. Thus, ESAT is
built as an actionable online artifact, allowing an easy access to its content. This approach is based on
Canca's (2020) “the box”2, a tool for the assessment of ethical principles for MLS.
Ethical approach and principles for the ESAT development
Given the theoretical discussion and requirements in the ISD domain described above, ESAT relies on the
applied ethics approach. Applied ethics is concerned with the practical application of moral
considerations to real world situations, as general moral theories and methods are often not specific
enough to be applied directly to concrete moral problems in a particular domain (Herwix et al., 2022).
Since earlier studies, e.g. (Chatterjee & Fuller, 2009), pointed out that governance built on different
ethical schools of thoughts may lead to contradictions and conflicts, ESAT provides solely an assessment
of the value realization. The assessment of and decision on the following action lies in the realm of the
auditor and thus complies to the individual weighting of the moral principles. ESAT is built on basic
generic deontological and teleological moral principles and theories (see e.g. (Rahanu et al., 2020) for a
review) and on the academic discussion on ethics in IT described above. The principles included in the
core framework in ESAT are: Autonomy, Harms and Benefits, Participation, Health/Well-being as well as
Efficient Use of Resources. These principles were chosen based on their significance in the debates
described above. Thus, ESAT incorporates an expandable framework for the assessment of the ethical
principles realized in an IT artefact. Also, different roles involved in ISD, such as software developer,
business user or individual user are reflected in the assessment process. During the assessment different
response paths can be taken, according to the role of the auditor.
Autonomy: Immanuel Kant’s moral philosophy states that, for an individual to be truly human, that
person must be free to decide what is in their best interest. From this understanding the principle of
autonomy emerges: The Kantian approach, the founding school of thoughts for the deontological
reasoning, affirms that someone has chosen an action freely. For such a choice to have significance, it
should be informed, that is, based on accurate information and an understanding of the issues at hand. If
this information is deliberately withheld or is incomplete because of carelessness, then the choice has
been made under false pretenses and thus is invalid. Accordingly, in ESAT, the value of autonomy is
broken down to the elementary values such as: transparency, content-privacy, explainability, accessibility,
changeability, privacy. Transparency of an information system allows an insight into its structure, data
collection, data processing or presentation (Floridi et al., 2020). Content- privacy extends this principle
by assessing what happens to the processed data after the interaction with the information system ends.
Explainability considers the question of how the presented information that might be shown to enhance
transparency is comprehensible in the given context (Tsamados et al., 2021). Accessibility addresses the
different modes of how the presented digital content can be retrieved in terms of different modes of
interaction with the information system. Changeability focuses on the variability of the system settings,
e.g. in terms of data use, use of the hardware resources or customization possibility for the user. Privacy
aspect focuses on the information about data collection and processing from the individual user
interactions with the system.
Harms and benefits: This value assesses the characteristic of the software that may lead to harmful
effects for the direct user or the stakeholders. The value is based on the principles developed for the MLS
assessment (Canca, 2020; European Commission, 2020). It is broken down into the values: security,
safety, impact on the relevant systems, accuracy/reliability and the use of dark patterns (Brignull, 2010).
Participation: Since the ISD team can use ESAT to assess their information system, participation is
included as a value. Assessment of this value should also reflect the involvement of stakeholders
mentioned in the VSD approach (Friedman, B., Kahn, P., Borning, 2002). In software development user
2
https://aiethicslab.com/the-box/
Twenty-ninth Americas Conference on Information Systems, Panama, 2023
3
Ethical Software Assessment Tool (ESAT)
participation is critically discussed as it might enhance the software acceptance (Bano & Zowghi, 2013),
but can also make the development process less effective (Subramanyam et al., 2010). Nevertheless, user
involvement is a widely accepted principle in development of usable systems (Kujala, 2010). Hence,
participation was included as part of its intrinsic value, also based on the Aristotelian notion of the good
life (Mulgan, 1990). The value of participation was operationalized in ESAT as: stakeholder participation,
requirements elicitation and periodization, accountability, and impact analysis. Whereas the definition of
accountability is taken from Dignum (2017) as being the explanation and justification of one’s decisions
and one’s actions to the relevant stakeholders.
Health and Well-being: Health effects associated with the use of computer technology has important
implications because of the prevalence of use of IT in various forms (Rahanu et al., 2020). The authors
consider involvement of health consideration in the IT design as part of a professional moral behavior.
They also stress the importance of considering the health aspect in ISD based on teleological and
deontological principles. Well-being is a complex concept, and there is little consensus on how to define it
(Dodge et al., 2012). Subjective well-being measures account for people’s cognitive and affective
evaluations. Rahanu et al. (2020) identify connection, community and belonging, recognition and
acknowledgement, self-expression, care, compassion, and empathy as contributing aspects to well-being.
They also argue that information systems can have both positive and negative effects on these aspects.
Thus, health and well-being are operationalized in terms of accessibility and changeability of user settings
among others in ESAT.
Efficient use of resources- sustainability: While sustainability can be measured using different
metrics, e.g. based on the 17 Sustainable Development Goals of the UN, the focus of ESAT lies on the
efficient use of the resources that are required to operate the assessed information system following the
values identified in VSD (Friedman et al., 2013). To operationalize this value, the use and adoption of the
sensors and data sources as well as other components of the hardware are included in the evaluation.
If the information system that is being assessed is a software that is used in a business context, further
potential ethical issues related to, e.g. the employee privacy (Vermanen et al., 2020), or vendor lock-in
(Nabbosa & Kaar, 2020) may arise. Thus, ESAT offers the assessment of this situation using the issues of
content-privacy, software induced habits (Jeyaraj, 2022) and workflow implementation.
Limitations and Outlook
This emerging research paper presented an online tool for ethical assessment of an information system.
While having the limitations of the testable prototype, this tools allows the examination on the use and
value of applied ethics tools in ISD. The evaluation of the concept for the tool usage without ethical
expertise is subject to future research. Nevertheless, the tool aims at providing an accessible realization of
the moral agency for the members of an information system development project or IT users. The
assessed ethical principles are grounded in ethical theory and ongoing ethical discussions, policies and
guidelines that are meant to govern the development of information systems. Application of ESAT does
not require installation of any specific software and the framework of the operationalized principles can
be extended according to the future development of legal and public requirements. The assessment results
show the status quo of the audited product. The subsequence actions and adjustments can then be
discussed and implemented in the ISD team or contribute to the ethical discussion. Future work will
include the evaluation of the tool application on the existing software as well as a guidance during the
design of new applications. To evaluate the usability and added value of the tool, close research of its use
among developers and designer through observations and interviews is planned.
References
Antolić, Ž. (2008). An example of using key performance indicators for software development process efficiency
evaluation. MIPRO 2008 - 31st International Convention Proceedings: Telecommunications and Information, 2,
156–161.
Bano, M., & Zowghi, D. (2013). User involvement in software development and system success: a systematic literature
review. ACM International Conference Proceeding Series, 125–130.
Brignull, H. (2010). Dark Patterns: Dirty Tricks Designers use to make People do Stuff.
Canca, C. (2020). Operationalizing AI ethics principles. Communications of the ACM, 63(12), 18–21.
https://doi.org/10.1145/3430368
Twenty-ninth Americas Conference on Information Systems, Panama, 2023
View publication stats
4