Table of Contents
Cover
Series Page
Title Page
Copyright Page
Dedication Page
PREFACE
ACKNOWLEDGMENTS
1 WHAT IS TECHNOLOGY (FROM AN ETHICAL POINT OF VIEW)?
1.1 A Hut in the Black Forest
1.2 The Question Concerning Technology: The Instrumental Theory of
Technology from Martin Heidegger to Joanna Bryson
1.3 “Post‐Phenomenology” and the Mediation Theory of Technology
1.4 Technologies Conceived of as Being More Than Mere Means or Instruments
1.5 Technologies Regarded as Moral Agents
1.6 Technologies Regarded as Moral Patients
1.7 Some of the Key Types of Technologies That Will Be Discussed at Greater
Length in Later Chapters of the Book
Annotated Bibliography
2 WHAT IS ETHICS? (AND, IN PARTICULAR, WHAT IS TECHNOLOGY
ETHICS)?
2.1 Two Campaigns
2.2 The Ethics of Virtue and Human Flourishing in Ancient Greece
2.3 Ancient Chinese Confucianism and Traditional Southern African Ubuntu
Ethics
2.4 Kantian Ethics
2.5 Utilitarianism and Consequentialist Ethical Theories
2.6 If Ethics More Generally Can Be All the Things Discussed in the Previous
Sections, then What Does this Mean for Technology Ethics in Particular?
2.7 How Technology Ethics Can Challenge and Create a Need for Extensions of
More General Ethical Theory
Annotated Bibliography
3 METHODS OF TECHNOLOGY ETHICS
3.1 Methodologies of Ethics?
3.2 The Ethics of Self‐Driving Cars
3.3 Ethics by Committee
3.4 Ethics by Analogy: The Trolley Problem Comparison
3.5 Empirical Ethics
3.6 Applying Traditional Ethical Theories
3.7 Which Method(s) Should We Use in Technology Ethics? Only One or Many?
Annotated Bibliography
4 ARTIFICIAL INTELLIGENCE, VALUE ALIGNMENT, AND THE
CONTROL PROBLEM
4.1 Averting a Nuclear War
4.2 What Is Artificial Intelligence and What Is the Value Alignment Problem?
4.3 The Good and the Bad, and Instrumental and Non‐Instrumental Values and
Principles
4.4 Instrumentally Positive Value‐Alignment of Technologies
4.5 Instrumentally Negative Misalignment of Technologies
4.6 Positive Non‐Instrumental Value Alignment of Technologies
4.7 Negative Non‐Instrumental Value Misalignment of Technologies
4.8 The Control Problem
4.9 Control as a Value: Instrumental or Non‐Instrumental? And Are There Some
Technologies It Might Be Wrong to Try to Control?
Annotated Bibliography
5 BEHAVIOR CHANGE TECHNOLOGIES, GAMIFICATION, PERSONAL
AUTONOMY, AND THE VALUE OF CONTROL
5.1 A Better You?
5.2 Behavior Change Technologies and Gamification
5.3 Control: Three Basic Observations
5.4 Key Dimensions of Control Discussed in Different Areas of Philosophy
5.5 Behavior Change Technologies and the “Subjects” and “Objects” of Control
5.6 The Value and Ethical Importance of Control
5.7 Concluding This Chapter
Annotated Bibliography
6 RESPONSIBILITY AND TECHNOLOGY
6.1 Two Events
6.2 What Is Responsibility? Different Ways in Which People Can Be Held
Responsible and Different Things for Which People Can Be Held Responsible
6.3 Responsibility Gaps: General Background
6.4 Responsibility Gaps Created by Technologies
6.5 Filling Responsibility Gaps by Having People Voluntarily Take
Responsibility
6.6 Should We Perhaps Welcome Responsibility Gaps?
6.7 Responsible Machines?
6.8 Human–Machine Teams and Responsibility
6.9 Concluding This Chapter
Annotated Bibliography
7 CAN A MACHINE BE A MORAL AGENT? SHOULD ANY MACHINES
BE MORAL AGENTS?
7.1 Machine Ethics
7.2 Arguments in Favor of Machine Ethics and Types of Artificial Moral Agents
7.3 Objections to the Machine Ethics Project
7.4 Possible Ways of Responding to the Critiques of the Machine Ethics Project
7.5 Concluding This Chapter
Annotated Bibliography
8 CAN ROBOTS BE MORAL PATIENTS, WITH MORAL STATUS?
8.1 The Tesla Bot and Erica the Robot
8.2 What Is a Humanoid Robot? And Why Would Anybody Want to Create a
Humanoid Robot?
8.3 Can People Act Rightly or Wrongly Toward Robots?
8.4 Can Robots Have Morally Relevant Properties or Abilities?
8.5 Can Robots Imitate or Simulate Morally Relevant Properties or Abilities?
8.6 Can Robots Represent or Symbolize Morally Relevant Properties or
Abilities?
8.7 Should We Be Discussing—Or Perhaps Better Be Avoiding—the Question
of Whether Robots Can Be Moral Patients, with Moral Status?
Annotated Bibliography
9 TECHNOLOGICAL FRIENDS, LOVERS, AND COLLEAGUES
9.1 Replikas, Chuck and Harmony, and Boomer
9.2 Ethical Issues That Arise in This Context Independently of Whether
Technologies Can Be Our Friends, Lovers, or Colleagues
9.3 Technological Friends
9.4 Technological Lovers and Romantic Partners
9.5 Robotic Colleagues
9.6 Are These All‐or‐Nothing Matters? Respect for Different Points of View
9.7 The Technological Future of Relationships
Annotated Bibliography
10 MERGING WITH THE MACHINE
10.1 The Experience Machine
10.2 Different Ways of Merging with—Or Merging with the Help of—
Technology
10.3 Transhumanism, Posthumanism, and Whether We Should Become—Or
Perhaps Already Are—Cyborgs
10.4 Some Critical Reflections on the Proposals to Merge with Technologies and
the Arguments and Outlooks Used in Favor of Such Proposals
10.5 Concluding Reflections: Revisiting the Hut in the Black Forest
Annotated Bibliography
Index
End User License Agreement
THIS IS PHILOSOPHY
Series editor: Steven D. Hales
Reading philosophy can be like trying to ride a bucking bronco—you hold on for
dear life while “transcendental deduction” twists you to one side, “causa sui”
throws you to the other, and a 300‐word, 300‐year‐old sentence comes down on
you like an iron‐shod hoof the size of a dinner plate. This Is Philosophy is the
riding academy that solves these problems. Each book in the series is written by
an expert who knows how to gently guide students into the subject regardless of
the reader's ability or previous level of knowledge. Their reader‐friendly prose is
designed to help students find their way into the fascinating, challenging ideas
that compose philosophy without simply sticking the hapless novice on the back
of the bronco, as so many texts do. All the books in the series provide ample
pedagogical aids, including links to free online primary sources. When students
are ready to take the next step in their philosophical education, This Is
Philosophy is right there with them to help them along the way.
This Is Philosophy: Second Edition
Steven D. Hales
This Is Philosophy of Mind: Second Edition
Pete Mandik
This Is Ethics: An Introduction
Jussi Suikkanen
This Is Political Philosophy: An Introduction
Alex Tuckness and Clark Wolf
This Is Business Ethics: An Introduction
Tobey Scharding
This Is Metaphysics
Kris McDaniel
This Is Bioethics: An Introduction
Udo Schuklenk
This Is Philosophy of Religion
Neil Manson
This Is Epistemology
Clayton Littlejohn and Adam Carter
This Is Philosophy of Science
Franz‐Peter Griesmaier and Jeffrey A. Lockwood
This Is Environmental Ethics
Wendy Lee
This Is Early Modern Philosophy
Kurt Smith
This Is Philosophy of Mind, Second Edition
Pete Mandik
Forthcoming:
This Is Aesthetics
Thi Nguyen and Nick Riggle
This Is Ancient Philosophy
Kirk Fitzpatrick
This Is Logic
Sara Uckelman
THIS ISTECHNOLOGY ETHICS
AN INTRODUCTION
SVEN NYHOLM
Copyright © 2023 by John Wiley & Sons, Inc. All rights reserved.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey.
Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by
any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted
under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written
permission of the Publisher, or authorization through payment of the appropriate per‐copy fee to the
Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750‐8400, fax (978)
750‐4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be
addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030,
(201) 748‐6011, fax (201) 748‐6008, or online at http://www.wiley.com/go/permission.
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc.
and/or its affiliates in the United States and other countries and may not be used without written permission.
All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated
with any product or vendor mentioned in this book.
Limit of Liability/Disclaimer of Warranty
While the publisher and author have used their best efforts in preparing this book, they make no
representations or warranties with respect to the accuracy or completeness of the contents of this book and
specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No
warranty may be created or extended by sales representatives or written sales materials. The advice and
strategies contained herein may not be suitable for your situation. You should consult with a professional
where appropriate. Further, readers should be aware that websites listed in this work may have changed or
disappeared between when this work was written and when it is read. Neither the publisher nor authors shall
be liable for any loss of profit or any other commercial damages, including but not limited to special,
incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our
Customer Care Department within the United States at (800) 762‐2974, outside the United States at (317)
572‐3993 or fax (317) 572‐4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not
be available in electronic formats. For more information about Wiley products, visit our web site at
www.wiley.com.
Library of Congress Cataloging‐in‐Publication Data applied for
Paperback ISBN 9781119755579
Cover Image: Wiley
Cover Design: Wiley
For Katharina
PREFACE
Suppose that a self‐driving car detects that a crash is unavoidable and that all
options open to it will involve harming human beings. What should a self‐
driving car do in such a situation? Is it ever permissible to use autonomous
weapons systems in warfare, which are specifically designed to kill human
beings and that select their own targets? What if a robot—e.g. a sex robot—is
designed to look and act like a human being? Does the resemblance mean that
we should treat that robot with any of the moral consideration we should show to
a human being?
What if the artificial intelligence we develop gets out of control? Who is then
responsible and what should we do about it? Are technologies always simply
value‐neutral tools? Or are they sometimes more than mere instruments? Are
human values—and human prejudices—sometimes embedded into our
technologies? Are the technologies we use extensions of ourselves? Have we
always been cyborgs? Should we merge with the technologies we use—for
example, by connecting our brains directly to the internet? What kind of future
should we aim to bring about with the help of the most advanced technologies at
our disposal?
These are the kinds of questions that are discussed within the ethics of
technology, or technology ethics, for short. These are the kinds of questions that
this book explores. The aim of the book is to introduce the reader to technology
ethics. The book aims to do so in a way that is accessible to readers who may not
have any previous familiarity with academic discussions of technology ethics.
This could be undergraduate students taking a course in technology ethics. Or it
could be anybody who just happens to be interested in this topic.
Researchers who work on technology ethics might hopefully also find this book
interesting and useful—for example, because it provides overviews of some of
the key ethical issues related to the technologies discussed in the book. There are
occasional deep dives into specific issues, which those with more background
might also find interesting. But the main aim is to introduce readers to
technology ethics, and to make them interested in exploring technology ethics
further. Hence the title This Is Technology Ethics: An Introduction.
This book is part of the series This Is Philosophy. The title of this series is
inspired by a series of very popular jazz albums that Columbia Records used to
put out. It was called This Is Jazz. Different albums in that series introduced
listeners to the greats of jazz, such as Miles Davis, Sarah Vaughn, Billie Holiday,
Duke Ellington, and so on. The aim was to give listeners who might previously
have been unfamiliar with jazz music samples of the best jazz performances by
great artists, such as the ones just mentioned and many others. This inspired
many people to go on and also listen to the deep‐cuts and explore the catalogs of
these artists in greater detail.
In the same way, the This Is Philosophy series aims to introduce readers to the
most interesting areas within philosophy, so as to inspire them to want to go on
to explore these parts of philosophy further. Other entries into the series include
a general introduction to philosophy called This Is Philosophy: An Introduction,
and books like This Is Philosophy of Mind, This Is Political Philosophy, and This
Is Metaphysics. There is a general introduction to ethics called This Is Ethics,
and also more specific ethics books such as This Is Bioethics, This Is Business
Ethics, and This Is Environmental Ethics. The current volume is similar to those
just‐mentioned more specific ethics books.
A title like This Is Technology Ethics can be read in two ways. It might be taken
to indicate that the book is covering all of what technology ethics is: the sum‐
total, so to speak, of what technology ethics is. Or it might be taken to indicate
that the book is offering samples of some of the key discussions and themes that
are part of technology ethics. This book does the latter. Accordingly, the phrase
“this is technology ethics” is here supposed to mean “these are some of the key
discussions and themes that are central within the field of technology ethics.”
Just like the albums in the This Is Jazz series did not mean to cover every
recording by the artists featured in the series, this book does not cover every
single issue within technology ethics. The book does aim to give readers a solid
idea of many of the most interesting parts of technology ethics. But there is more
to technology ethics than we will be able to cover in our exploration of it below.
The topics that will be covered in what follows are among the most fascinating
parts of technology ethics.
Another choice an author of a book like this faces is the following. One can try
to write in a way that is as neutral as possible, which does not present any new
arguments and does not offer any opinions or make any judgments.
Alternatively, one can offer an “opinionated introduction” to something. That is,
one can introduce a topic while also sometimes taking a stance on some of the
issues, offering arguments and views of one's own, so as to enter into a dialogue
with the reader. This book takes the latter approach. In other words, this is an
opinionated introduction to technology ethics.
The idea is that if a philosophy book is not neutral but instead sometimes offers
some opinions and arguments, it is likely to be more engaging than a book that
tries to be completely neutral at all times. The reader is not only free but also
invited to disagree with any and all of the ideas in the book. One of the main
points of any work in philosophy—including an introductory book like this one
—is to make the reader think for him or herself. One way of achieving that aim
that often works is to try to provoke the reader a little bit. Accordingly, the book
tries to do that here and there—but always in a way that is intended to be
friendly and respectful toward the views and ideas discussed throughout the
book.
Some contemporary technologies—such as the social media platforms many of
us use—encourage short and snappy exchanges, where emotions often run high
and people sometimes get angry at each other and start insulting one another.
This is not the type of debate this book takes as its model. One of the wonderful
things about philosophy is that it is possible to have deep disagreements and
offer arguments against one another’s views, while doing this in a way that is
friendly and respectful toward all those involved in the discussion. That is the
model this book aims to emulate.
I also mention this example about how contemporary technologies may inspire a
certain form of behavior for another reason. It helps to illustrate one of the ideas
that will be a running theme throughout the book: namely, that the technologies
we use influence and shape us, sometimes in ways we find positive, but
sometimes also in ways that we may find negative upon reflection. On social
media platforms, for example, one sometimes sees people expressing regret that
discussions that start on such platforms so easily escalate into non‐productive,
angry exchanges.
That raises another issue discussed in technology ethics: is it unethical to create
technologies that may lead some people to act in morally problematic ways? Or
should the fact that many other users will use these technologies in harmless or
positive ways be taken to outweigh the risks involved in introducing new
technologies that affect how people interact with each other, whether it is online
or offline?
Those are some more examples of what technology ethics is about. And those
are hopefully examples that help to illustrate that not only is technology ethics
relevant to all of us. It is probably also something most of us have already
thought about, if not explicitly under the heading of “technology ethics,” then at
least implicitly. It is very hard to not think about technology ethics issues when
one watches the news, or when one simply goes about one's daily life in a world
with so much modern technology everywhere.
This book tackles these kinds of issues in an explicit way. It spells out some of
the main issues that those interested in technology ethics tend to spend their days
thinking about—or perhaps lie awake at night worrying about. This book is
about robots, artificial intelligence, autonomous weapons systems, self‐driving
cars, brain implants, information and communication technologies, technologies
related to love and sex, technologies for medical treatments or for human
enhancement, technologies that might change what we are as human beings or
what we can become, and more. Those are the sorts of things the book will be
discussing from an ethical point of view. This is technology ethics.
ACKNOWLEDGMENTS
My work on this book has benefited greatly from conversations and exchanges
with, among others, the following people: Joel Anderson, Antonio Bikić, Jan
Broersen and the MA students in our 2021 core seminar on the philosophy of AI,
Joanna Bryson, Mark Coeckelbergh, John Danaher, Brian Earp, Arzu Formenek,
Lily Frank, Cindy Friedman, John‐Stewart Gordon, David Gunkel, Caroline
Helmus, Ziagul Hosseini, Naomi Jacobs, Geoff Keeling, Maximilian Kiener,
Marjolein Lanzing, Kritika Maheshwari, Giulio Mecacci, Anna Melnyk, Kęstutis
Mosakas, Jilles Smids, Joshua Smith, Daniel Tigard, Darja Vrščaj, Lucie White;
the students in my technology ethics course and other courses; the ethics
researchers involved in the Human Brain Project; and my colleagues in the
Ethics of Socially Disruptive Technologies research program in the Netherlands.
Part of my work on this book was supported by the Gravitation grant program of
the Dutch Ministry of Education, Culture, and Science and the Netherlands
Organization for Scientific Research (NWO grant number 024.004.031).
Some of this material has been presented at workshops and events organized by
Lukas Brand, Niël Conradie, Olle Häggström, Tomi Kushner, Julia van der
Linde, Janina Loh, Wulf Loh, Leo Menges, Saskia Nagel, Anna Puzio, Markus
Rüther, and Arleen Salles, as well as at colloquia and conferences at the
following universities: the University of Amsterdam, Delft Technical University,
Eindhoven University of Technology, Karlsruhe Institute of Technology, LMU
Munich, the University of Münster, the University of Oxford, the University of
Porto, Utrecht University, and the University of Vienna. My thanks to the
audiences at those events for their helpful feedback. I am also grateful to three
anonymous peer reviewers of the whole book manuscript, whose feedback was
very beneficial to me while I was preparing the final version of the manuscript.
Many thanks also to the editor of this book series, Steven Hales, as well as to
Laura Adsett, Mandy Collison, Charlie Hamlyn, Vinitha Kannaperan, Marissa
Koors, Will Croft, Hannah Lee, and the other members of the team at Wiley‐
Blackwell who have been involved in this book project.
Furthermore, I have benefited from the support of my wife and our families in
Sweden and Germany. This book is dedicated to my wife, Katharina Uhde, with
love.
1
WHAT IS TECHNOLOGY (FROM AN ETHICAL
POINT OF VIEW)?
1.1 A Hut in the Black Forest
1.1 Around 100 years ago, in the Black Forest of Southern Germany, there stood
a small and simple three‐room cabin, to which an eccentric philosopher would
retire in order to escape from the modern world. From 1922 onward, he went
there to work on philosophical texts about the nature of “being,” and he felt
deeply inspired by these surroundings. The philosopher in question was Martin
Heidegger. He composed some of his most well‐known writings in this cabin,
which he called “die Hütte” (the hut). While in his Hütte, Heidegger would
sometimes dress up in traditional farmers’ clothing. And he would wander about
in the Black Forest and ponder abstract philosophical questions, far away from
bothersome modern technologies and other distractions.
1.2 This might seem like a strange place to start this book, and a strange choice
of philosopher. However, it makes sense to start a book about technology ethics
with Heidegger and his hut for a number of reasons. One reason is that one of
the most‐cited writings about what technology is was written by Heidegger
(most likely in the cabin!). Another reason is that it is good to have a reminder
early on about the following. Even though we usually think about the latest and
most advanced forms of cutting‐edge technology when we hear the word
“technology,” even the simplest and oldest forms of technology are also
technologies.
1.3 For example, Heidegger would go and get water from a well when he visited
his hut. (There was no electricity or running water when he first started
frequenting his cabin.) The well where Heidegger got his water is a technology,
which can perform a very important purpose, just as something like a self‐
driving car or the latest smartphone is a technology. Heidegger himself tended to
romanticize the past. He made a sharp distinction between modern technologies
(of which he was skeptical) and more traditional technologies (which he happily
embraced). But when we think about the general question of what technology is,
we should keep in mind that there are many different kinds of technology, both
old and new. Old technologies are technologies, just like new ones are.
1.4 If a traditional water well is a technology just as much as a self‐driving car or
a smartphone is a technology, then what exactly is a technology? This question
—“what is a technology?”—is what we will focus on in this first chapter. Since
this is a book about technology ethics, it is important to have an idea of what we
should understand by “technology” in the first place. The ancient philosopher
Aristotle recommended that one should always start by defining one’s key terms.
That certainly seems like good advice.
1.5 It will also be important to try to define or explain what we mean by
“ethics,” so that we can put “technology” and “ethics” together to form the idea
of “technology ethics.” But we will save the question of what ethics is until the
next chapter and here focus on what technology is. When we do so, however,
that will already start raising ethical issues even before we get around to trying
to say what ethics is, as we will do in the next chapter. One reason for this is that
when one tries to explain important concepts, the choices we make about how to
define these concepts might prove themselves to be controversial, and raise
questions about what is important or valuable.
1.6 For example, consider the questions “what is a human being?” and “what is a
person?” On the face of it, these might not seem like inherently ethical
questions. But depending on who or what you think qualifies as a human being
or person and for what reasons, you might find that before you know it, there
will be others who find your views and their implications to be highly
controversial. Ethical controversies about abortion, for example, tend to turn
partly on who or what counts as a human being or as a person.
1.7 Choices about whose views about something to listen to—e.g., about what
“technology” should be taken to be—can also be controversial and raise ethical
questions. Heidegger, for example, has in recent times become a highly
controversial figure. While he was rector of the University of Heidelberg during
the German Nazi era, Heidegger was a member of the Nazi party. That by itself
does not necessarily mean that Heidegger was a Nazi. Anybody in a leading
position may have needed to be a member of that party during that era. But the
recently discovered “Black Notebooks” (some notebooks of Heidegger's) have
revealed that for a while, Heidegger was not only a member of the Nazi party but
also convinced of some of the ideological ideas associated with Nazism. Some
even think that Heidegger's romanticism and his musings about the simple life
close to nature and his hut in the forest are somehow connected with Nazi ideals.
1.8 Heidegger scholars claim that Heidegger fairly quickly came to reject the
ideas of the Nazis. Furthermore, the essay “The Question Concerning
Technology” that is of relevance here was written after Heidegger abandoned
any sympathies he had with Nazi ideas for a while. Yet it might seem
controversial to start a discussion of what technology is with the ideas about this
topic from a one‐time Nazi. Nevertheless, as noted above, Heidegger's just‐
mentioned essay is one of the most‐often discussed contributions to the main
topic of this chapter. So, it is good to be aware of some of the things he says
about what technology is, for the sake of context. Moreover, as we will see
below, there are also philosophical reasons to start with Heidegger's discussion,
despite whatever flaws Heidegger himself might have had as a person, at least
during certain stages of his life.1
1.2 The Question Concerning Technology: The
Instrumental Theory of Technology from Martin
Heidegger to Joanna Bryson
1.9 The further into Heidegger's “The Question Concerning Technology” one
gets while reading it, the harder the text becomes to understand. But there are
some bits at the beginning of that essay that are fairly straightforward. For
example, let us consider this passage:
Everyone knows the two statements that answer our question. One says:
Technology is a means to an end. The other says: Technology is a human
activity. The two definitions of technology belong together. For to posit ends
and procure and utilize the means to them is a human activity. The
manufacture and utilization of equipment, tools, and machines, the
manufactured and used things themselves, and the needs and ends they serve,
all belong to what technology is. The whole complex of these contrivances is
technology.
(Heidegger 1977: pp. 4–5)
A couple of lines down, Heidegger adds:
The current conception of technology, according to which it is a means and a
human activity, can therefore be called the instrumental and anthropological
definition of technology.
(Heidegger 1977: p. 5)
Notice that what Heidegger does here is in effect to first formulate two different
theories of technology, and to then combine them into a hybrid theory. The
instrumental theory of technology defines technologies as tools or instruments
that human beings use as means to their ends. The anthropological theory of
technology defines technologies as parts of distinctively human activities. The
combined theory (“the instrumental and anthropological definition of
technology”) represents technologies as means to ends within human activities.
1.10 Another thing that Heidegger by implication also does is to highlight that
we can think about technology from different points of view. The instrumental
theory of technology, for example, fits with what one might call an engineering
mindset. Engineers love to identify concrete problems, and to then seek out or
design the best tools for solving these problems. From such an engineering point
of view, it is very natural to think about technologies as tools used as means to
the end of solving concrete problems. The anthropological theory, in contrast,
adopts the mindset of an anthropologist. Anthropologists and other social
scientists study human practices and activities. So, it is natural for them to
portray technologies as parts of human practices or activities.
1.11 Another noteworthy thing about the instrumental theory of technology, as it
is usually interpreted, is that it represents technologies as being inherently value‐
neutral. Think about the saying “guns don't kill people, people do.” That slogan
has been used by the American National Rifle Association as a response to the
charge that having lots of guns in society heightens the risk of violence. The idea
behind “guns don’t kill people, people do” seems to be that guns are in
themselves value‐neutral tools. It is people with bad intentions who might use
these tools in bad ways. It is also possible to use these value‐neutral tools in
good ways, according to this part of the instrumental theory of technology.
1.12 In general, any tool can be put to good or bad uses, according to this way of
seeing things. Accordingly, if bad things happen and technologies are involved,
we should not blame the technologies, but the people who use them for their
own ends, which might be morally problematic. On the flipside, when
technologies are used for good ends, the instrumental theory implies that we
should not thank or praise the technologies but the people who create or use
them. Technologies can deserve neither praise nor blame, according to this way
of thinking. That might seem obviously true. But as we will see below, there are
those who take a different view.
1.13 Speaking of how technology can be interpreted from different points of
view, below we will consider how we should define or understand technology
when we take up a distinctively ethical or perhaps more broadly philosophical
point of view. But let us first linger just a little bit longer with the instrumental
theory of technology, setting aside the anthropological perspective on technology
for a moment. In particular, it is worth noting that even though a purely
instrumental theory is often criticized in contemporary technology ethics, there
are also prominent defenders of the instrumental theory of technology in current
discussions. By considering one example, we can illustrate how the instrumental
theory of technology can (1) be used in ethical arguments and (2) be further
extended.
1.14 Joanna Bryson is a computer scientist and roboticist who has recently
transitioned into the role of a professor of ethics. Among other things, she is the
author of a striking essay entitled “Robots Should be Slaves” and a number of
follow‐up articles that further refine the argument in her original piece.2 In her
characteristically forceful language, Bryson applies an updated version of the
instrumental theory to the question of whether we should ever regard robots with
artificial intelligence as worthy of rights or moral consideration.
1.15 Bryson often tells an anecdote about how she came to be interested in this
topic that helps to illustrate what she is concerned about. When Bryson worked
in a robotics lab at MIT early in her career, there was a robot called “Cog” that
people in the lab used to talk about in ways that suggested that they owed moral
consideration to it. They would say things such as “don't pull the plug or turn
Cog off—that would be to kill Cog!” Occasionally, they would say such things
without realizing that Cog the robot was actually already unplugged. These
people were projecting humanlike qualities onto a robot.
1.16 As Bryson sees things, this way of behaving around robots is a big mistake.
Robots—like any other technologies—are tools that we create for our human
purposes. Moreover, and here comes the key dimension that Bryson adds to the
instrumental theory of technology: robots and other technologies are the property
of people. They can be bought and sold. Since robots are property that people
can buy and sell and tools we use for human purposes, technologies are like
“slaves” or “servants” we own, just like some human beings were once regarded
as tools, who could be bought and sold, in some societies of the past.
1.17 In the case of human beings, Bryson thinks that it is wrong and horrible that
there were ever slaves anybody could own: tools that others could buy and sell.
But that, Bryson thinks, is what all human‐created technologies are and should
be. One must add “should be” here, because Bryson has the interesting view that
it is possible to create robots or other artificially intelligent technologies toward
which we could have duties, but that we should avoid doing so.
1.18 If we could create machines that can experience suffering or that are
intelligent and sensitive in the ways that human beings and many animals are,
then Bryson thinks we would have obligations towards these creations. But we
should avoid creating such robots. We should only create technologies that it is
okay to treat as tools, which we can buy and sell. Since Bryson adds these ideas
about what should and should not be done to the instrumental theory, we can say
that she presents a normative version of the instrumental theory of technology.
1.19 One of the real‐world robots and related events Bryson has reacted strongly
to is the robot Sophia and the 2017 event where this robot was given honorary
citizenship of the Kingdom of Saudi Arabia. Sophia is a robot that looks like a
human being (a “humanoid robot”). The robot has a humanlike face and is able
to imitate human speech. The back of the robot's head is transparent, so that one
can see that it is a machine. But the robot is put in many distinctively human
situations: for example the robot has appeared on talk shows such as The Tonight
Show with Jimmy Fallon. And many human beings—including highly influential
people—have treated the robot as if it were a human person. For instance, the
former Chancellor of Germany, Angela Merkel, once took a “selfie” photograph
together with Sophia. Moreover, the robot has been invited to speak in front of
high‐profile international political bodies, like the United Nations and the
Munich Security Council.
1.20 For somebody like Bryson, who adopts a normative version of the
instrumental theory of technology, the just‐described events are highly
problematic from an ethical point of view. In an interview, Bryson commented
on this in the following way:
What is this about? It's about having a supposed equal you can turn on and
off. How does it affect people if they think you can have a citizen that you
can buy?3
Again, on the instrumental theory of technology, a robot and any other
technology is a tool, which is value‐neutral in itself, and that you can buy or sell.
Since Bryson subscribes to that view, she thinks that the way people behave
around Sophia the robot is highly problematic. However, not everyone agrees
with the instrumental theory of technology. Let us now explore some other ideas
about what technology is or can be.
1.3 “Post‐Phenomenology” and the Mediation
Theory of Technology
1.21 As was mentioned above, we can adopt different mindsets when we reflect
on what technology is or should be taken to be. One thing we can do is to ask
what we should think of technology as being from an ethical point of view. For
most of the rest of this chapter, that is the point of view from which we will
consider what technology is. In other words, our question will be: “when we
think about technology from the point of view of ethical theorizing, what should
we then understand technology as being?” Taking up this ethical point of view
on technology, many authors have criticized the purely instrumental theory of
technology.
1.22 Some sociologists and philosophers who consider what technology is, from
an ethical point of view—and who criticize the instrumental theory— do so in a
way that fits with what Heidegger calls the anthropological way of
understanding technology. That is, they are looking at and analyzing human
practices involving technologies. They are interested in how the technologies we
have around us affect us. And they argue that within many human practices,
technologies are not always merely tools that are completely value‐neutral in
nature. Technologies, they argue, can also play other roles in our lives. One such
group of thinkers are the members of the so‐called post‐phenomenological
school of thought.
1.23 The term “post‐phenomenology” is not only hard to pronounce. It may also
be hard to understand for those who have not heard it before. So, let’s try to
break this idea down into parts. The first thing to know is that “phenomenology”
was a philosophical movement that started around the time that Heidegger was
writing his philosophy in his hut. This movement was premised on the idea that
in reflecting on human life, we should not start with abstract philosophical
theories, but with the phenomena that we encounter in the world. In our
philosophizing, we should be investigating how reality and how we ourselves
appear to us from the point of view of how we experience things as human
beings. Post‐phenomenology takes this idea a step further and asks how
technologies shape how we experience reality, ourselves, and what we are able
to do or aim for.
1.24 Members of this school of thought—such as Don Ihde, Bruno Latour, and,
more recently, Peter‐Paul Verbeek—are interested in how technologies
“mediate” the way we experience things and what we are able to do.4
Technologies, they say, are a medium in between us and what we perceive and
what we can do. There are two different ideas here. One has to do with inputs,
the other with outputs. Let us consider them one after the other.
1.25 One key idea is that technologies shape how we perceive or experience the
world and even ourselves. To use a simple example, if we are wearing glasses,
our visual experiences and perceptions are going to be different than if we
remove our glasses. Or, to use another example, if you visit the Louvre in Paris
and you view the Mona Lisa through the camera on your phone, as many tourists
who visit the Louvre do these days, then your perception of the Mona Lisa will
be mediated or shaped by your phone. It will be different from what it would
have been if you simply look straight at the painting without your phone in
between it and your eyes. More generally, what we pay attention to as we go
about our lives is shaped by the technologies we use. For example, these days,
what news (or fake news!) people hear about and pay attention to tends to be
strongly influenced by algorithms that are part of the social media that they use.
1.26 The second key idea is that technologies shape what we are able to do and
how we are able to do it. Relatedly, technologies shape what we see ourselves as
being able to do. They also sometimes “suggest” certain actions to us. For
example, these days, we can take weekend trips to faraway places because we
have airplanes that can quickly take us to a distant country. So, the idea of taking
a weekend trip to some place far away might occur to a person, because the
technologies available to us make this possible. This would not have been
possible, nor anything that would have occurred to anybody to do, before these
technologies were available. Speaking of social media—to return to the example
at the end of the previous paragraph— that is also an example of a technology
that might shape how we act, for example how we communicate with other
people.
1.27 A lot of people would never think of, or even dare to say certain things to
other people face‐to‐face that they think of saying and dare to say to them on
social media. This can be both good and bad: good because people might dare to
speak out against something, bad because people might hurl abuse at each other.
This is another example of how technology shapes what we do and what we
think of doing.
1.28 Actually, there is an idea in Heidegger's “The Question Concerning
Technology” essay that is very much in line with this post‐phenomenological
idea that technologies shape what we can do and how we perceive the world.
Heidegger talks about how new technologies—e.g., a water powerplant—might
change how we view something like a river. Suddenly, the river is presented by
the modern water powerplant as a source of electricity, or as a means to an end to
which it could not have been viewed as a means before.
1.29 Heidegger takes this idea to the extreme. He argues that the more modern
technologies we surround ourselves with, the more nature will appear to us to be
a large set of means to human ends as opposed to an end in itself. While a river
might have appeared to us as a beautiful piece of nature before, a new
technology might suddenly present it as a means to the end of producing
electricity. As a result, the way we perceive the river might then have been
changed by the technology.
1.30 This general idea is picked up by post‐phenomenologists when they
criticize the part of the instrumental theory of technology that portrays
technologies as being in themselves value‐neutral. Members of this school of
thought argue that technologies are not purely value‐neutral for the reason that
technologies influence how we value things around us. Peter‐Paul Verbeek, for
example, is fond of using the example of ultrasound images of fetuses. Verbeek
claims that such images present fetuses in a new light (different from the mere
knowledge that there is a fetus in the womb). In particular, those images present
the fetus as a patient, whose health status the parents need to worry about and
make decisions about.
1.31 Another type of example of how technologies are not value‐neutral tools
according to post‐phenomenologists is often illustrated with the help of an idea
that Don Ihde discusses. The idea is that technologies sometimes contain
intended or unintended “scripts.” That is to say, they may tell us what to do and
perhaps even force us to act in certain ways. For example, the French sociologist
and philosopher Bruno Latour talks about speed bumps as telling drivers to slow
down, paper cups as communicating to users that they should be thrown away
after use, and the beeping noises cars make when we do not put on our seatbelts
as forcing us to wear our seatbelts.
1.32 Members of this post‐phenomenological movement also criticize the
instrumental theory of technology by arguing that the instrumental theory tends
to falsely represent humans and technologies as being wholly separate from each
other when it portrays humans as acting on the basis of ends and technologies as
being means used to these ends. This is not correct, many post‐
phenomenologists argue. It is not correct because human beings and
technologies form “assemblages” or units that can do things that humans alone,
without the technologies, neither could nor would do on their own.
1.33 Verbeek, for example, takes issue with the above‐mentioned idea that “guns
don’t kill people, people do.” According to Verbeek, a man with a gun easily
becomes what Americans tend to call a “gunman,” i.e., somebody with a gun
and a disposition to shoot. The man and his gun form a unit, as Verbeek sees
things. And this unit can do things, and may become inclined to do things, that
the man cannot or would not do without his gun.5
1.34 Notably, the science and technology researcher Donna Haraway suggests
that is helpful to think of us human beings using the metaphor of cyborgs. There
is, as Haraway sees things, no sharp distinction between humans and
technologies. Many of us have heard the expression “you are what you eat.”
Haraway's view seems to be something along the lines of “you are, in part, the
technologies you use.” The idea that human beings might merge with
technologies, or that we have already done so, is something we will have
occasion to return to later in this book. Post‐phenomenologists are not the only
ones fascinated by that idea. As we will see in Chapter 10, many others are
fascinated by it as well.
1.35 To summarize the key points in this section: according to post‐
phenomenologists, technologies are not value‐neutral tools that are wholly
separate from human beings, who set ends and use technologies as means to
their ends. Technologies are more than that. Technologies shape how we
perceive the world, what we are able to do, and what we value and how we value
it. Humans and the technologies we use are not wholly separate from each other.
They form units that are able to do things, and that are also disposed to do
things, that humans without these technologies would neither be able nor
inclined to do. This can be seen as an updated and refined version of the
anthropological theory of technology since this is a theory about what roles
technologies play in human practices. These interesting ideas from the post‐
phenomenologists will be popping up here and there throughout this book. But
let us set them aside for now, and return to the instrumental theory of technology
once more.
1.4 Technologies Conceived of as Being More
Than Mere Means or Instruments
1.36 Recall how Joanna Bryson talks about how robots and other technologies
are, and should be conceived of as, tools that we own, which we can buy or sell,
and which should be treated as means to human ends. This brings to mind some
ideas that the illustrious Enlightenment philosopher Immanuel Kant discusses in
his classic, very influential ethical treatise Groundwork for the Metaphysics of
Morals. What Bryson says about how we should treat technologies can be seen
as an inversion, or the opposite, of what Kant says about how we should treat
human persons. Since Bryson talks about technologies and Kant talks about
persons, their views can be seen as complementing each other.
1.37 In that just‐mentioned book, Kant divides up the world into two separate
categories: what he calls “persons” and “things.” Persons can think and act, be
rational, and make moral decisions. They have an absolute value and dignity, and
should be treated with care and respect. Persons, in Kant's phrase, should be
regarded as “ends‐in‐themselves.” They can be responsible for their actions, and
are members of the moral community. Everything else in the world, Kant argues,
belongs to the category of things. Anything that is not a person only ever has a
relative value—namely, a value relative to the desires or wishes of persons.
1.38 As Kant sees things, all things can be treated as mere means to the ends of
persons. Persons, in contrast, should always be treated as ends and never as mere
means. Kant also relates this to our distinctive humanity. When he does so, Kant
formulates the following well‐known and very popular moral principle: “so act
that you always treat the humanity in each person, never merely as a means, but
always at the same time as an end‐in‐itself.” This is sometimes called the
“formula of humanity” or, alternatively, Kant's principle of “persons as ends‐in‐
themselves.”6
1.39 We can restate Bryson's normative version of the instrumental theory of
technology in these Kantian terms in the following way: unlike human beings,
all technologies are and should be regarded as things, and not persons. They
should always be treated as mere means and never as ends‐in‐themselves.
Moreover, we should avoid creating any technologies that would ever be
anything other than mere things that we can treat as mere means. We should
never create technologies that are or appear to be persons, which are ends‐in‐
themselves. Bryson’s normative instrumental theory of technology, in other
words, can be seen as an interesting inversion of Kant's principle of humanity.
1.40 It makes sense to bring up this Kantian terminology and these Kantian ideas
here, not only because they are interesting in themselves but also because they
give us a philosophical vocabulary with which we can reinterpret the
instrumental theory of technology. It also makes sense to bring up these things
because Kant's philosophy, just like Heidegger's, has had a lot of influence
within technology ethics, as we will see later on.
1.41 Moreover, it is also worth bringing up these ideas from Kant and their
relation to Bryson's updated version of the instrumental theory of technology
because of certain striking recent developments within the ethics of technology.
Recently, several technology ethicists have defended ideas that clash starkly with
pretty much all of the ideas just mentioned in the paragraphs above. In particular,
some technology ethics researchers have recently defended one or more of the
following ideas:
Some technologies can think and act, and be rational and make moral decisions.
Some technologies can be persons, as opposed to mere things.
Some technologies should be treated as ends (i.e., be shown moral consideration
or given rights), and not as mere means.
Some technologies can be responsible for what they do, just like human persons
can be responsible for what they do.
1.42 Such ideas can seem counterintuitive. They are perhaps shocking and even
disturbing to those who accept any version of the instrumental theory of
technology. But as noted, these ideas are gaining a foothold within contemporary
technology ethics. To see why some philosophers of technology are beginning to
take them seriously, we can now consider some examples of how some people
interact with new forms of technologies like robots and other technologies
equipped with artificial intelligence. We will save a full discussion on these
controversial ideas until later chapters. But it makes sense to briefly also look at
them in this first chapter, since we are here considering the question of what we
should understand by the idea of technology.
1.5 Technologies Regarded as Moral Agents
1.43 We have already briefly considered the example of how some of Bryson's
colleagues talked about the robot “Cog” (e.g., they were hesitant about pulling
the plug or turning off the robot). And we have also considered how some people
have been treating Sophia the robot (e.g., giving honorary citizenship to the
robot, inviting it to appear on talk shows, taking selfie photos with it, or allowing
it to speak in front of important political bodies). These are examples of people
treating robots in ways that suggest that they regard them, or that they are
willing to treat them, as being more than mere means or things that are
categorically different from persons. Many more examples can be given. And
more examples will be given throughout the book. But here are just a few more
examples of how people think about technologies in ways that clash with the
purely instrumental theory.
1.44 Many people working on questions in technology ethics are interested in
what are sometimes called functionally “autonomous” machines. That is,
machines that for some period of time are able to operate on their own,
performing specific tasks, without direct human steering. Such technologies are
occasionally put into situations where they seem to need to be able to make
moral decisions. Notably, many philosophers and other researchers think that it
is possible to create machines that can make moral decisions. A widely discussed
example is that of the self‐driving car. Another is the example of military robots
or what are sometimes called autonomous weapons systems. These technologies
operate in contexts that are dangerous and where human lives are at stake.
1.45 We can imagine that a self‐driving car might need to “decide” whether to go
left or right when a fork in the road occurs. The breaks of the car may have
stopped working. The problem might be that each option involves a threat to
human beings. On the road on the left, there might be five people, who would
not be able to get away from the road in time and thereby avoid being hit by the
car. On the road on the right, there might be one person, who would also not be
able to get away from the road in time to avoid being hit by the car. The person
riding in the car may have fainted, and therefore not be able to offer any input
regarding what to do. So, it might seem that this self‐driving car, whose breaks
are currently not working, needs to make a “moral decision.” Should it go left,
and potentially injure or even kill five people? Or should it go right and thereby
save the five, but potentially injure or kill one person? This is widely regarded as
a type of situation in which a machine—in this case, a self‐driving car—needs to
be able to make a moral decision and then act on it.
1.46 An interdisciplinary research field called “machine ethics” is devoted to
investigating whether it is possible to create moral machines and if so, how this
can and should be done. According to this way of thinking, the technologies in
question are not simply regarded as value‐neutral tools that are used by human
beings to achieve their ends. Rather, the technologies are regarded as what
philosophers call “moral agents.” That is, they are regarded as entities or beings
capable of making moral decisions and acting on those decisions.
1.47 Most ethicists of technology who take this idea seriously do not think that
the technologies in question would be moral agents of the sorts that we human
beings are. Rather, the technologies—the self‐driving cars, military robots, or
whatever it might be—would be some other kind of moral agent. They might be
able to make moral decisions and act on them, but perhaps they would not be
morally responsible for their decisions in the ways that human beings can be.
This is a common position. It is defended, for example, by the well‐known
Italian technology ethicist and information philosopher Luciano Floridi.7
1.48 As Floridi sees things, any entity that can act in some sense and that can act
in situations that are morally significant is a moral agent. But this is not enough,
Floridi thinks, for regarding the entity as a morally responsible agent who can be
blamed or praised for its actions or decisions. This idea of artificial agents that
are not able to be responsible but that are nevertheless making morally sensitive
decisions is sometimes thought to give rise to worries about so‐called
responsibility gaps. This expression refers to the idea that a machine, e.g. a
military robot, might autonomously make a morally important decision and do
something (e.g., kill a human being), but then not be able to be held responsible
for the outcome. There may also not be any human beings who can be held
morally responsible for what the robot did. At the same time, it might seem as if
somebody should be held responsible for what happened. Hence a gap in
responsibility apparently arises. Worries about such responsibility gaps are
widely discussed within technology ethics. And most ethicists of technology do
not think that these problems can be solved by holding the machines themselves
responsible.
1.49 There are others, however, who take the more extreme view that some
technologies can not only make moral decisions, but sometimes also be
responsible for those decisions in some sense. The philosopher Daniel Tigard,
for example, has argued in a series of academic articles that there are many
different “faces of moral responsibility” (that is, many different aspects of what
is involved in being morally responsible). According to Tigard, some advanced
technologies, such as some artificially intelligent robots or other autonomous
systems, can be responsible in certain, at least limited, ways.8 A view such as
Tigard's is very far removed from a purely instrumental view of technology, on
which all technologies are value‐neutral tools or means to human ends. If we
think that some technologies can be responsible for what they do or decisions
they make, our understanding of technology is very different from the sort of
view Bryson advocates.
1.50 We have just considered technologies in the role of what philosophers like
to call “moral agents.” There are also examples where technologies seem to
occupy a role that philosophers call that of “moral patients.” A moral patient is
an entity or being towards which it is possible to have obligations or duties, or
that it is possible to treat more or less well. In short, a moral patient is a being or
entity with moral status. It is an entity or being that might have rights of some
important sort. When Sophia was made into an honorary citizen of Saudi Arabia,
for example, this might have seemed to bestow certain rights and some
important form of social status onto Sophia, since citizens are typically regarded
as having certain rights and an important status. But let us now also consider
another example to illustrate the idea of robots as potential moral patients.
1.6 Technologies Regarded as Moral Patients
1.51 In February of 2015, the news network CNN published an article with the
headline “Is it cruel to kick a robot dog?”9 This article was about some
interesting reactions to a video that had been posted by a tech company called
Boston Dynamics. The video featured a robot that looks a little bit like a dog,
and which is very good at keeping its balance. The robot, called “Spot,” is
shown running on a treadmill and walking up some stairs, doing all this without
falling over. Later in the same video, and again in order to illustrate how good
the robot is at keeping its balance, one of the Boston Dynamics engineers is
shown kicking Spot. Sure enough, Spot is able to keep its balance and does not
fall over, even after a pretty solid kick from the engineer. While Spot was able to
stabilize itself after being kicked, many viewers of this video lost their cool
when watching Spot being kicked. CNN reported that viewers reacted to the
video with comments such as: “Kicking a dog, even a robot dog, just seems so
wrong,” “Poor Spot!,” and “Spot getting kicked is creepy.”
1.52 One thing that is fascinating about these reactions, from a philosophical
point of view, is that the people who reacted in this way to this video seemed to
regard Spot the robot dog as more than a mere tool, which we can treat in any
way we might want for the sake of our ends. The people who reacted in this way
seemed to be responding in a way that suggests that they are willing to regard
this robot as having some form of moral standing. Their reactions portrayed Spot
as a moral patient, i.e., as a being or entity toward which one can act wrongly
and that should be treated with some moral consideration.
1.53 But not only people who post comments to online videos adopt that stance
towards technologies. Some ethicists of technology have started to take seriously
the idea that some technologies—such as some robots—should be regarded as
moral patients, to which we should give moral consideration. There are even a
few scholars who are taking seriously the idea of what they call “robot rights.”
Some of them even go as far as to discuss the idea of “robotic persons.”
1.54 For example, Mark Coeckelbergh and David Gunkel have developed what
they call a “relational” theory of moral status ascriptions. What they mean by
this is that we should consider the question of who or what deserves to be treated
with moral consideration, or be given rights, as being a question of what sorts of
relationships human beings can have with those entities. Just like non‐human
animals can become part of human communities and thereby acquire special
moral status and even rights (e.g., as pets or farm animals do), so it could also be
that some technologies (e.g., some robots) might become part of human
communities. It might then make sense to extend moral consideration, and give
rights, to them. Or so Coeckelbergh and Gunkel argue.
1.55 If we move away from the instrumental theory of technology and we adopt
this relational way of thinking about human‐technology interaction, we would
take what Coeckelbergh and Gunkel call a “relational turn” in our thinking about
technology. On such a view, what technologies themselves are able to do is seen
as less important than what relations there might be between these technologies
and the humans interacting with them. Sophia the robot, for example, might not
have human emotions or be a sensitive or intelligent being. But a robot like
Sophia might nevertheless become part of human society and thereby become
something akin to a person with moral standing and even certain rights.10
1.56 Another type of theory that has been suggested to defend the idea of
extending moral status to robots is what the Irish legal academic and moral
philosopher John Danaher calls “ethical behaviorism.” According to this idea, if
a technology consistently behaves like a human being or an animal that has
moral standing behaves, then we should treat that technology with the same
degree of moral consideration that we show for a human being or an animal. For
example, if a piece of technology behaves in a way that indicates that it is
suffering (for instance, screaming in apparent pain), then since we should avoid
causing unnecessary suffering to human beings and animals, we should also
refrain from whatever actions are causing the technology to behave as if it is
suffering.11
1.57 Danaher has even gone so far as to extend this idea of ethical behaviorism
to the topic of friendship. He argues that if a technology—e.g., a robot, an avatar,
or a chatbot—can consistently behave in the way that a friend behaves, then we
can be friends with these technologies, just like we can have human beings as
our friends. More on that in Chapter 9.
1.58 These are all examples of ordinary people and researchers who think that
some technologies are not to be regarded as only being tools or instruments that
are value‐neutral in themselves and that we can always use as mere means for
whatever ends we might have. These are examples of non‐instrumental ways of
regarding some technologies, and of ways of attributing a humanlike status to
some types of technology. In other words, they are examples of how, from an
ethical point of view, it is possible to understand what technologies can be in a
wide and perhaps surprising range of different ways, some of which are very far
removed from the purely instrumental theory of technology.
1.59 The ideas and examples surveyed in this section are so controversial—and
so interesting!—that we cannot just leave them hanging like this. We will discuss
all of them in much more detail later on in the book. One whole chapter will be
devoted to whether technologies can be moral agents (Chapter 7). Another
chapter will be devoted to whether technologies can be moral patients (Chapter
8), and yet another to whether technologies can be our friends or have other
significant relationships with human beings (Chapter 9). For now, the main point
of briefly introducing these examples has simply been to illustrate that there is a
wide range of ways of regarding what technologies can be and what roles they
can have within our lives. The instrumental theory of technology provides one,
but not the only possible perspective on what roles technologies can have.12
1.7 Some of the Key Types of Technologies That
Will Be Discussed at Greater Length in Later
Chapters of the Book
1.60 Up until this point, we have focused on the very general question of what
technology is or what it could be. We have seen that depending on what mindset
one adopts when one thinks about technology, one can take radically different
views about what technologies are or can be. For example, if one puts on one's
engineering hat, one is likely to view technologies as tools that we design and
use to solve specific problems, whereas if one puts on one's anthropologist hat,
one might think of technologies as parts of human practices or activities. And if
we think about what technologies are from an ethical point of view, we have
seen that there are even those who think that some technologies can be moral
agents or moral patients, or have the status as some form of persons, or perhaps
even become our friends.
1.61 So far, the discussion has been on a pretty general level. We have asked
what “technology” refers to in general terms. At this point, some might respond
—not without some justification!—that it might be better to try to explain or
define certain specific classes of technology that are particularly interesting from
an ethical point of view. So that is the last thing this first chapter will do.
1.62 We can start with the concept of a robot, a concept that has already been
used a number of times above. What is a robot? Some—such as the robot ethicist
David Gunkel—are unwilling to define what a robot is. There are so many
different kinds of machines that are classified as “robots,” and some of them
may seem to have little to do with each other. But if pressed, researchers like
Gunkel are likely to define what a robot is with reference to what is sometimes
called the “sense, think/plan, act” paradigm. According to this way of seeing
things, a robot is a machine with sensors and actuators, which is able to take in
information about its environment, process that information, and take some form
of action in response to this information about its environment—and which does
so in the service of specific tasks that the robot is supposed to perform.
1.63 For example, a vacuum cleaning robot like the “Roomba” robot is able to
detect whether there are big pieces of furniture in its path. And it is able to use
this information from its sensors to steer itself in a different direction, to thereby
be able to carry out its task of vacuuming the room it is operating in. When we
think of robots, we do not only think of robots like the Roomba robot, however.
We might also think of paradigmatic robots out of science fiction, that is, robots
that look a little bit like metallic or silvery beings with a roughly humanlike
form, which may move and talk in a “robotlike” way.
1.64 If functional robots like Roomba or lawn‐moving robots, or logistics robots
or military robots, etc., are on one end of a spectrum, and paradigmatic robots
out of science fiction are at the middle of the spectrum, we can place so‐called
humanoid robots at the other extreme end of the spectrum. These are robots
created to look or act like human beings. Sophia the robot is one example.
Another example is a robot created by the Japanese robotics researcher Hiroshi
Ishiguro. This robot is a replica of Ishiguro himself, which looks and, to some
extent, behaves like Ishiguro. In general, then, a robot is a machine that is
embodied, that can sense its environment, and that can respond to the
environment based on the information it takes in, so as to perform some task or
set of tasks.
1.65 When robots or other technologies are able to imitate or replicate human
behaviors in a seemingly intelligent way, they are sometimes said to have
“artificial intelligence” or “AI,” in short. In other words, a technology or
technological system is said to possess artificial intelligence when it is able to
perform tasks that humans (or animals) use their intelligence to perform. In the
most widely used textbook about AI, written by Stuart Russell and Peter Norvig,
a two‐fold distinction is made, which is worth repeating here. On the one hand,
Russell and Norvig distinguish between technologies that are meant to either
behave or think in intelligent (or intelligent‐seeming) ways. On the other hand,
they distinguish between technologies attempting to reach a human level in their
performance of some task and technologies designed to reach an optimally
rational level in their performance of their tasks. According to this way of
thinking, there could be machines behaving at a human level, thinking at a
human level, or perhaps behaving or thinking at a more optimally rational level
than humans are able to.
1.66 We will have occasion to discuss different forms of robots and AI
throughout the book. But one last introductory observation might be of interest
here. Namely, while the term “robot” was introduced in science fiction, the term
“artificial intelligence” comes from a scientific context. The word “robot” comes
from a 1920 stage play by the author Karel Čapek, in which artificial humans
(“robots”) are created to work in a factory. In contrast, the term “artificial
intelligence” was introduced by a team of scientists, in a research proposal from
1955. The scientists set themselves the goal of creating machines that would be
able to perform tasks that we human beings need our intelligence to perform.
1.67 In short, then, the term “robot” comes from fiction, whereas the term
“artificial intelligence” comes from science. But in both cases, our ideas about
these things tend to be influenced and colored both by science and by science
fiction, and sometimes primarily by the latter. So, it is good to be careful and to
be aware of how strongly influenced we might be by ideas from fiction rather
than from science when we think about technology ethics. This might not be a
bad thing. Science fiction has produced many interesting ideas and concepts. But
it can mean that our expectations of what robots and AI can do or be might
sometimes be overly optimistic, if not outright unrealistic.
1.68 Another kind of technology that will be of interest in some of the chapters
to come include (self‐)tracking and data‐logging technologies, which might also
be so‐called behavior change technologies. These are technologies that are used
to track, log, or otherwise gather and make use of data and information that
might be generated by our actions or behaviors, whether online or offline. Some
of these technologies might be designed to influence or change the ways we
behave, by stimulating additional motivation to act in some way, for example by
creating incentives or other prompts to change our behavior. Technologies can
do so, for example, by gamifying certain behaviors or activities. This means that
they turn something that is ordinarily not conceived of as a game into a game or
something game‐like. In other words, game‐like elements are introduced into
some otherwise not necessarily game‐like activity or domain.
1.69 A running app that one might have on one's phone can explain all three of
these ideas. Such an app is likely to track information/data about how much,
how often, where, and how fast the user is running. This information, and
information about how other people run, might then be used in a way that
gamifies the user's running: there might be points given, achievements declared,
or a leaderboard where the user can see how they are doing in comparison to
other runners. This will make exercising by running more like a game. And it
might influence and motivate the user to run more often, run longer distances, or
run faster.
1.70 Information and communication technologies will also be important
throughout the book, as well as in any discussion about technology ethics.
Broadly, this includes any forms of technologies we use to seek information or to
communicate with those around us, such as search engines, social media
platforms, or the devices we use for such purposes, such as computers or
smartphones. Importantly, some information and communication technologies
might be partly gamified, and they might track behavior, collect data about us,
and so on. In general, from an ethical point of view, the idea of data‐collecting
technologies—i.e., technologies that collect data about people for different
purposes—is a controversial form of technologies. They are ethically
controversial since users may not know what will happen to their data, and since
having their data might give others (such as big tech companies) some control or
power over them.13
1.71 We will also occasionally be discussing medical technologies (i.e.,
technologies used for medical purposes or perhaps broadly having to do with
health) in distinction to technologies for other domains of life. Some
technologies may start as medical technologies—or, say, as military technologies
—and then start also being used in other domains, for other purposes than those
originally intended. This is a common occurrence. Ethicists of technology often
worry about the potential for what is sometimes called the “dual use” of
technologies: they may have been designed and intended for one domain (e.g.,
some medical application), but then be co‐opted for use in some other domain as
well (e.g., some questionable military use), which the inventors or designers of
the technology may not be fully on‐board with. This is related to worries about
unintended consequences of the use of the technologies, another topic that we
will have plenty of occasions to return to in the chapters that follow.
1.72 Another form of technology that is worth mentioning here as a fascinating
kind of technology are brain stimulation technologies or brain implants. These
are technologies used to target particular brain regions, such as, for example, the
sort of deep brain stimulation technologies that is used in one form of treatment
of Parkinson's disease and other neurologically based afflictions. This is an
example of a kind of technology that might have been developed for medical
use, but that might conceivably later be put to other uses as well, instead of the
ones for which the technologies were originally designed.
1.73 These just‐mentioned different types of technologies—robots, AI, brain
implants, and so on—are all quite different than the traditional forms of
technologies that Heidegger philosophized about and had in his hut, which we
visited at the beginning of the chapter. But, as was already noted at the beginning
of the chapter, we should not give in to the temptation of only regarding these
kinds of high‐tech emerging technologies as technologies. As noted above, older
technologies are also technologies, just like newer ones are.
1.74 Actually, Heidegger makes another point about technologies that is worth
mentioning as we bring the discussion in this chapter to a close. He notes that
when technologies are working well and we have become accustomed to them,
they sort of fade into the background and become almost transparent to us. We
do not really notice them. Instead, as the post‐phenomenologist philosophers
note, we start to perceive reality through our technologies and act through our
technologies as well. It is typically only when technologies stop working
properly or when new technologies are introduced and when we are not yet used
to them that we notice them and that they are in the foreground of our
experience. We quickly come to take technologies for granted. This is something
worth keeping in mind as we discuss ethical questions related to different
technologies.
1.75 As noted earlier on, what we will do in the next chapter is to consider the
question of what ethics is or what it should be taken to be. It turned out in this
chapter that explaining what technology is isn't as straightforward as one might
think. Even if Heidegger was of the opinion that everyone will agree with the
instrumental and anthropological definition of technology, we have seen that
there can be quite a bit of disagreement about how one should think about what
technology is. We will see in the next chapter that the same applies to ethics. It is
one of those concepts that when we try to define it, things quickly become
contentious, and disagreements arise.
1.76 In the case of ethics, this is to be expected. Ethics is a normative enterprise.
It has to do with what is good and bad, right and wrong, meritorious or shameful,
and so on. And it is in the nature of everything normative that people tend to
disagree about it. So, it is with ethics in general. And so it is with technology
ethics in particular. This is part of what makes it interesting.
Annotated Bibliography
Sarah Bakewell (2017). At the Existentialist Café: Freedom, Being, and Apricot
Cocktails with Jean‐Paul Sartre, Simone de Beauvoir, Albert Camus, Martin
Heidegger and Others, London: Other Press.
An accessible introduction to the ideas of, among others, Martin Heidegger
and the phenomenologists. Contains more information about Heidegger's hut,
as well as about his controversial time as a member of the Nazi party.
Joanna Bryson (2010). “Robots Should be Slaves,” in Close Engagements with
Artificial Companions, edited by Yorick Wilks, Amsterdam: John Benjamins
Publishing Company, pp. 63–74.
An oft‐quoted contemporary defense of the instrumental theory of
technology, with a particular focus on robots.
David Gunkel (2018). Robot Rights, Cambridge, MA: The MIT Press.
A deep‐dive into the literature about non‐instrumental perspectives on robots.
Discusses the idea of treating robots as persons and giving them moral
consideration.
Martin Heidegger (1977). The Question Concerning Technology, and Other
Essays, New York: Garland Publishing.
Not exactly an easy read, but the most accessible of Heidegger's writings, and
a bit of a classic within the philosophy of technology.
Peter‐Paul Verbeek (2011). Moralizing Technology: Understanding and
Designing the Morality of Things, Chicago: Chicago University Press.
A defense of the post‐phenomenological perspective, including the mediation
theory of technology, by one of its leading exponents.
Notes
1 The Wikipedia entry on “Martin Heidegger”
(https://en.wikipedia.org/wiki/Martin_Heidegger) features a picture of
Heidegger's hut:
https://en.wikipedia.org/wiki/Martin_Heidegger#/media/File:Heideggerrundw
eg0009.JPG
2 Bryson has expressed some regret about the title of her paper. She now prefers
talking about robots as servants we own rather than as slaves we own. But the
argument remains the same. You can hear Bryson discuss her ideas in this
episode of the “Philosophical Disquisitions” podcast: “Episode #24—Bryson
on Why Robots Should Be Slaves”:
https://philosophicaldisquisitions.blogspot.com/2017/06/episode‐24‐bryson‐
on‐why‐robots‐should.html
3 James Vincent (2017) “Pretending to Give a Robot Citizenship Helps No
One,” The Verve, https://www.theverge.com/2017/10/30/16552006/robot‐
rights‐citizenship‐ saudi‐arabia‐sophia
4 This short video, by Verbeek, introduces the mediation theory of technology:
“Animation: Explaining Technological Mediation”:
https://www.youtube.com/watch?v=FVhrLwBNbvU
5 You can hear Verbeek discuss these ideas in more detail in this episode of the
“New Books in Philosophy” podcast: “Peter‐Paul Verbeek: Moralizing
Technology: Understanding and Designing the Morality of Things”:
https://newbooksnetwork.com/peter‐paul‐verbeek‐moralizing‐technology‐
understanding‐and‐designing‐ the‐morality‐of‐things‐university‐of‐chicago‐
press‐2011
6 Here is a mini lecture on Kant's humanity formula from the “Cogito” channel:
“Kant's Second Categorical Imperative”: https://www.youtube.com/watch?
v=0Cl0FG‐xUKA.
You can listen to a longer discussion about Kant's ethics on this episode of
BBC's radio program “In Our Time” here: “Kant's Categorical Imperative”:
https://www.bbc.co.uk/sounds/play/b0952zl3
7 Here is an interview with Floridi, from “The Dissenter”: “#329: Luciano
Floridi: Information, Knowledge, Science, and AI”:
https://www.youtube.com/watch?v= jgmdEk7rTxU. Floridi discusses his
views about artificial moral agents and responsibility just after the 38:03
mark.
8 You can hear Tigard discuss this topic on this episode of the “Philosophical
Disquisitions” podcast: “79: Is There a Techno‐Responsibility Gap?”
https://philosophicaldisquisitions.blogspot.com/2020/08/79‐is‐there‐techno‐
responsibility‐ gap.html
9 Phoebe Park (2015) “Is It Cruel to Kick a Robot Dog?” CNN Edition:
https://edition.cnn.com/2015/02/13/tech/spot‐robot‐dog‐google/index.html
10 You can listen to Gunkel talk about his ideas on the “Dolores Project” podcast
here: “Robot Rights w/David Gunkel”: https://anchor.fm/joshua‐k‐
smith6/episodes/Robot‐Rights‐w‐David‐Gunkel‐e12viq7
11 You can listen to an audio essay by Danaher on “ethical behaviorism” here:
“Assessing the Moral Status of Robots: A Shorter Defence of Ethical
Behaviourism”:
https://philosophicaldisquisitions.blogspot.com/2019/10/assessing‐moral‐
status‐of‐robots.html
12 For references to the work mentioned in this and previous section, see the
annotated bibliographies of Chapters 7–9.
13 In an episode of the “Philosophy 247” podcast, you can hear the philosopher
Carissa Véliz being interviewed about ethical issues related to data‐collecting
technologies: “The Future of Privacy”:
https://philosophy247.org/podcasts/the‐ future‐of‐privacy/
2
WHAT IS ETHICS? (AND, IN PARTICULAR, WHAT
IS TECHNOLOGY ETHICS)?
2.1 Two Campaigns
2.1 The “campaign to stop killer robots” was launched in 2013. It is a response
to the potential development of fully autonomous weapons that would “decide
who lives or dies, without further human intervention.” That is how this
campaign presents its aim on their website. They are opposed to autonomous
weapons, the website explains, because such technologies would lack key human
characteristics such as compassion and the human capacity for moral judgment.
2.2 According to this campaign, the development of weapons systems with
significant autonomy threatens to create a “destabilizing robotic arms race.”
Moreover, replacing human soldiers with machines might make the decision to
go to war easier. The machines might make terrible mistakes, and civilians
would have to suffer the consequences. The use of these weapons is likely to be
disproportional. And it will be unclear who could be held responsible for any
unlawful acts that might be caused by fully autonomous weapons systems. Such
an “accountability gap would make it difficult to ensure justice, especially for
victims.”
2.3 Lastly, the campaign claims that these kinds of weapons might also be used
in other problematic ways, outside of armed conflicts, such as to suppress
peaceful protests or for excessive border control. For all of these reasons, the
campaign to stop killer robots suggests the following solution to the problems
associated with autonomous weapons systems: the “development, production
and use of fully autonomous weapons must be banned.” This could be done by
national laws and international treaties (http://stopkillerobots.org).1
2.4 Another campaign, which is influenced by the campaign to stop killer robots,
is the “campaign against sex robots.” This campaign was started by the computer
science professor and anthropologist of technology Kathleen Richardson in
2015. More recently—in 2021—the campaign changed its name to the
“campaign against porn robots.” Richardson and her fellow campaigners think
that it is impossible to have sex with a robot. One can only have sex with another
person. So, it is better, they argue, to speak about “porn robots” rather than “sex
robots,” which is a more common name for robots created specifically for sexual
purposes.
2.5 On the campaign's website, the campaigners declare that their number 1 goal
is to “abolish pornbots in the form of women and girls.” (What, one might ask,
about sex robots in the form of men or boys, or in the form of non‐binary
people?) In support of their aim to ban sex robots, the campaign says that it
endorses the following values: “humanity,” “connections,” “ethics,” “potential,”
and “love.” The campaign also refers its website's visitors to the 2015 position
paper “The Asymmetrical Relationship: Parallels between Prostitution and the
Development of Sex Robots” by Richardson.
2.6 In that paper, Richardson argues that sex robots are likely to reinforce
negative stereotypes about sex partners, in particular about women. She also
argues that there is a danger that objectifying attitudes that users of sex robots
might have toward sex robots might carry over to, or reinforce objectifying
attitudes toward, human women. In a recent video that explains the name change
mentioned above, Richardson also expresses a concern that sex robots might
encourage and glamorize rape and child abuse.2 Therefore, the development of
sex robots should be abolished (http://campaignagainstsexrobots.org).
2.7 These two campaigns represent one conception of what technology ethics is
about. They represent a conception of technology ethics as being about
arguments for why certain technologies or technological developments should be
forbidden or banned—along, perhaps, with justifications for why certain
controversial technologies should be permitted or tolerated. When some people
—including some people from the tech industry—hear the word “ethics,” this is
the conception of ethics that comes to mind for them. It is about what one is
allowed to do or not allowed to do, and arguments for or against forbidding or
allowing certain things. To be sure, this is an important part of ethics more
generally and also an important part of technology ethics in particular. However,
if we stop there and settle for the idea that this is the whole of what technology
ethics is about, we will be left with an overly narrow idea of what it is all about.
2.8 As already announced in the first chapter, the topic of this second chapter is
the question of what ethics is. By looking at what ethics is more generally, and
then putting our findings about that together with what we learned about
different conceptions of technology in Chapter 1, we can arrive at an idea about
what technology ethics is. The conception of technology ethics that we will
arrive at below will inform the rest of the chapters in the book. A key aim of this
chapter is to present a broader conception of what ethics is than the above‐
mentioned conception according to which it mainly has to do with what should
be forbidden or permitted.
2.9 To illustrate that technology ethics in particular and ethics more generally are
about more than arguments for or against forbidding or permitting things, this
chapter will survey a range of different approaches that people have taken to
ethics, both during the history of this subject and in current debates. The chapter
will cover a lot of ground and consider a wide range of different topics and ideas
associated with ethics. But it will not be able to cover everything that falls under
the broad umbrella of “ethics.” Accordingly, technology ethics as a subject can
be conceived of as being even broader and more varied than the broad
conception that will be presented in this chapter.
2.10 Another thing that this chapter will do in its final section—and that the rest
of the whole book will also do—is to try to illustrate the following points. Issues
that come up in technology ethics can require us to rethink some of our
traditional ways of thinking about ethics. They can also challenge some of the
ways that we think about ourselves and what we are as human beings. Moreover,
technology ethics can require us to reflect on and perhaps even reconsider what
is most important to us in life.
2.11 The targets of the two above‐mentioned campaigns help to illustrate these
last points. For example, if warfare can be outsourced to technologies like
autonomous weapons systems, and human beings might therefore no longer need
to perform the role of soldiers, this may fundamentally challenge many ideas
traditionally associated with the ethics of warfare. As noted above, autonomous
military technologies can also create challenges for our ethical ideas about
responsibility for deaths in wartime and the question of who should be
accountable for them. In general, any autonomous technologies that might make
life‐and‐death decisions raise many questions about whether we should
reconsider our ideas about moral responsibility and what it means to be a moral
agent. More on this in Chapters 6 and 7.
2.12 Consider next the prospect that robots might enter into the domain of sex—
and perhaps also into the domain of loving relationships. Or suppose that
modern technologies more generally (i.e., not just robots) become even more
entangled into the areas of sex and love than they might already be. This might
force us to reassess how we think about intimate human relationships. And it
might challenge the values we associate with them. Later on, we will devote a
whole chapter to that particular topic. We will discuss this in more detail in
Chapter 9. For now, though, the main aim of the present chapter is to consider
what ethics is to begin with. So, let us get right to it.
2.2 The Ethics of Virtue and Human Flourishing
in Ancient Greece
2.13 When one is considering what ethics is, there are few more fitting places to
turn to than Ancient Greece. The term “ethics” derives from the ancient Greek
term “ēthika,” which means customs. The term “morality” derives from Latin,
and also means customs or mores. So while some people sometimes talk about
“ethics and morals”—and some people, including some philosophers, distinguish
between ethics and morality—“ethics” and “morality” pretty much mean the
same thing. Or at least these words both come from words from the Ancient
world that used to mean the same thing.
2.14 Ancient Greece was a hotspot for philosophy. Indeed, it was the birthplace
of the tradition that is sometimes called “Western philosophy.” Ancient Greece
also was the home of a veritable who's who of philosophers most people have
heard of. It is mind‐boggling to consider that three of the most famous
philosophers ever—Socrates, Plato, and Aristotle—were all immediately
connected with each other. Socrates was a teacher of Plato's, and Plato was a
teacher of Aristotle's. Not a bad lineage!
2.15 Ancient Greece is the birthplace of written philosophy in the Western
tradition. (We'll get to two other traditions in the next section, in case readers
were wondering whether there would only be discussion of Western philosophy.)
Socrates himself did not write anything. He simply hung out, if you will, with
other philosophically interested people, discussing philosophy with them,
making quite an impression on many people, who then wrote about him.
2.16 Plato, for example, featured Socrates as a main character in most of his so‐
called Platonic dialogues. Those dialogues are a sort of philosophical plays. The
characters are typically philosophers discussing philosophical topics such as
“what is knowledge?”, “what is virtue?”, “can virtue be taught?” or “is virtue
knowledge?” If readers have not read Plato's Republic, for example, that is a
dialogue that is focused on the questions “what is justice?” and “what is good
about being just?” But it also covers just about every topic in philosophy, and
contains some of the most famous ideas and arguments in the history of
philosophy. It is well worth a read, and many re‐reads! We will consider some
ideas from Plato's Republic later.3
2.17 Here, however, we will not focus on Plato, but rather on Aristotle. We will
do so because Aristotle's ethical theorizing—especially as represented in his
book the Nicomachean Ethics—is highly representative of what we today tend to
associate with ethics in Ancient Greece and in the Greco‐Roman world. Key
themes are questions such as the following: what is the highest good in life (i.e.,
what should we aim for in life)? What is it to flourish or live well as a human
being? What virtues or excellences do we need to flourish and live well? How do
we develop these virtues and excellences? And what is friendship and why is
friendship a great good?
2.18 That those are the sorts of things that Aristotle discusses in his most well‐
known ethical treatise helps to illustrate the point made above that ethics is not
only about what should be forbidden or permitted. In the ethical treatises and
discussions from the Ancient Greeks—as well as from the Ancient Roman
authors who were inspired by the Ancient Greeks—those narrower questions
played a fairly small part, though they were of course also of some importance.
What was of the greatest importance was the question of what it is to live a good
life.
2.19 Aristotle claims that human beings all tend to strive after what he calls
“eudaimonia.” This word is sometimes translated into “happiness,” but usually
with a footnote explaining that “happiness” in the modern sense is something
narrower than what eudaimonia is. Eudaimonia involves taking pleasure in what
one is doing and in how one's life is going (similar to happiness in the modern
sense). But it also includes the broader idea of living well or flourishing as a
human being.
2.20 As Aristotle sees things, different kinds of beings differ with respect to
what is involved in flourishing or doing well as those kinds of beings. For
example, what it is to live well or to flourish as a dog or a cat or a horse is
different than what it is to live well as a human being. Why? Because human
beings have different capacities and a different potential to develop certain
virtues and excellences than dogs, cats, or horses do. So, Aristotle thinks that we
must ask what is distinctive of us human beings. Once we have an idea about
this, this gives us an idea of what it is to live well or flourish as a human being.
2.21 Aristotle thinks that human beings are “rational animals” and “political
animals,” meaning that we have reason and intelligence and that we live in social
communities. So, what it is to live well as a human being, according to Aristotle,
importantly involves developing our capacities for intellectual and social virtues.
Virtues, within Aristotle's theory, are habits or dispositions to act and think in
certain ways. In a phrase that is sometimes used, virtues are dispositions to “do
the right things, for the right reasons, in the right kinds of situations.”
2.22 These virtues, the theory continues, tend to be middle‐paths between
different extremes: e.g., being brave is a middle‐path between being cowardly
and being foolhardy. Moreover, another idea within Aristotle's theory—and also
many other theories from this period—is that virtues form an interlocking
system, so that, for example, one can only have the virtue of being a just person
if one is also a wise, prudent, and temperate person, and so on. This is
sometimes dubbed the “unity of the virtues” thesis.
2.23 Another noteworthy thing about virtues, Aristotle observes, is that they tend
to be met with admiration. They tend to inspire others to want to associate with
or become friends with those who appear to have the virtues. There are different
kinds of friendship, according to Aristotle, including what he calls “utility
friendships” and “pleasure friendships.” But the highest and “most perfect”
friendship is what he calls “virtue friendships.” Such friendships come about
when people who possess important human virtues are drawn to each other, form
bonds of mutual esteem and shared values, and then develop together into better
people because of their association based on the virtues and excellences that they
have and then develop further together.
2.24 We can highlight one last thing about Aristotle's theory of virtue: namely,
that he thinks that one way of acquiring virtue is to emulate role models or
exemplars that are wise and virtuous and to thereby acquire the habits and
dispositions of the virtuous. This idea was picked up by various other schools of
thought, some of whose names have become familiar parts of everyday
language: the Skeptics, the Stoics, the Epicureans, the Cynics, and other schools.
For all of them, Socrates was a role model and an exemplar. Their philosophies
tried to distill the essence of what it is to be a Socrates‐like person. And they
asked what virtues and habits one would need to adopt to become such a person.
2.25 In a slightly simplified analysis of these schools of philosophy, one can say
that they all formulated ideas of what the “highest good” for a human being is,
what one would need to do in order to attain the highest good, and what role the
virtues had in all of this. The differences among these schools of thought are
sometimes fairly subtle.
2.26 The skeptics and epicureans both, for example, praised peace of mind/inner
peace as the highest good, but had somewhat different ideas about how to
achieve this goal. The skeptics recommended adopting a generally skeptical
attitude, whereas the epicureans recommended a simple life filled with simple
pleasures. Virtue, for the epicureans, was a means to the end of tranquility of
mind. In contrast, for the stoics and the cynics, virtue was an end in itself, and
part of the “highest good” for a human being. They understood virtue, in part, as
being constituted by self‐control and wisdom. And they then had related but
subtly different ideas of how to attain self‐control and wisdom.
2.27 In short, the main focus of ethical reasoning among the Ancient Greeks was
not what should be forbidden and what might be permitted, but rather the
exploration of the best possible life for a human being, the role of virtue and
excellence in such a life, and the quest for self‐knowledge and wisdom.4 An
interesting fairly recent development in technology ethics is that some
technology ethicists are trying to incorporate these ideas about flourishing,
virtues, and the emulation of role models into their philosophizing about
technology.
2.28 For example, in a book that came out around the same time that this book
was being written, Berndt Carsten Stahl argues that the ethics of artificial
intelligence should be based around the idea of human flourishing. (See the
annotated bibliography.) And as we will see later, some technology ethicists
have begun discussing whether it is possible to have what Aristotle calls virtue
friendships with AI‐driven technologies such as chatbots and robots. More on
that in Chapter 9.
2.29 Let us now consider some non‐Western traditional forms of ethics, so that
we do not only look at the Western tradition. Turning to other traditions for
insights about different ways of thinking about ethics is another thing that
technology ethics scholars have recently started doing more and more. So, it
makes sense for us here to also briefly look at some non‐Western ideas. The two
examples we will briefly consider—ancient Chinese Confucian ethics and
traditional southern African ubuntu ethics—can also be interestingly compared
with the ancient Greco‐Roman ethics we have just had a look at.
2.3 Ancient Chinese Confucianism and
Traditional Southern African Ubuntu Ethics
2.30 Above, we saw that whereas Socrates did not write any philosophical works
himself, others wrote about him. He ended up becoming an almost mythical
figure and revered role model for generations of ancient Greek philosophers
after he had died. Speaking of Socrates' death, it has also become the stuff of
legends since Socrates famously died by voluntarily drinking a cup of poison
after having been put on trial for “corrupting the youth” with his philosophizing.
2.31 Just as Socrates was the mythical figure and key role model of ancient
Greek philosophy, Confucius was a key mythical figure and role model of
ancient Chinese philosophy.5 He was, if you will, the Socrates of ancient China
—not only in the sense of becoming a role model, but also for the reason that no
writings by Confucius himself have been preserved. Rather, the information we
have about the thought of Confucius comes from writings by others. The key
source of information about Confucius and his philosophizing is a text known as
The Analects. The authors of the Analects are also unknown. But they have had
an immense influence on East Asian intellectual and social history, just like
writings about Socrates have had a great influence on the Western tradition of
philosophy.
2.32 So, what are some distinctive features of Confucian ethical thinking? One is
a focus on the creation and maintenance of a harmonious social order. This
applies both to society in general, and to family life. For example, the idea of a
harmonious social order involves ideas about the duties we have in certain roles,
such as in the role of a parent or in the role of a child with parents toward whom
we have certain duties in our role as their children. Accordingly, loyalty is an
important value from this point of view.
2.33 Associated with these ideas about creating and maintaining a harmonious
social order there are also ideas about the value of having and performing certain
rituals, which help to strengthen and establish a harmonious social order. Rituals,
from this point of view, are seen as both valuable in themselves and as
instrumental to maintaining a harmonious social order.
2.34 Another key theme is that of self‐cultivation or self‐improvement. At birth,
we are all equal in our potential, according to the Confucian perspective.
However, depending on how we cultivate ourselves, some develop their potential
to become “profound persons,” whereas others become “small” persons (or what
one might call “narrow‐minded”):
The profound person understands what is moral. The small person
understands what is profitable.
(Analects, 4.16)
The profound person develops a character that encompasses virtues such as
compassion and humaneness—and, over the course of their life, wisdom. The
profound person, thus understood, thereby becomes greater than the narrow‐
minded person. In a poetic formulation a little later in the Analects, the authors
muse that:
The moral force of the profound person is like the wind; the moral force of
the small person is like the grass. Let the wind blow over the grass and it is
sure to bend.
(Analects, 12.19)
Notably, a version of the Golden Rule (“do unto others as you would have them
do unto you”) is also attributed to Confucius in the Analects. Confucius' version
of this principle goes like this: “What you do not desire for yourself, do not do to
others” (Analects, 15.24). This maxim is one of the things that is supposed to
guide one's self‐cultivation when one is trying to develop one's moral character
over time, according to this way of thinking.
2.35 In short, Confucian ethics puts a high value on a harmonious social order. It
emphasizes duties tied to roles people have, prizes rituals, and recommends life‐
long self‐cultivation, whereby people can grow to become profound persons.
This is a fascinating approach to ethics—which, as noted earlier, has recently
been incorporated into contemporary technology ethics by some writers.
2.36 One example is the 2021 book Harmonious Technology: A Confucian
Ethics of Technology, which is a collection of papers that was edited by Pak‐
Hang Wong and Tom Xiaowei Wang. Most technology ethics publications have
tended to build on philosophy in the Western tradition. But it is only one
tradition among others. So, it is important that philosophers like Wong and Wang
—and the contributors to their book—are taking steps to incorporate ideas from
non‐Western traditions into technology ethics. After all, as Wong and Wang note,
modern technology is a global phenomenon. So, it seems only right that
technology ethics should try to develop a global perspective as well.
2.37 Speaking of incorporating ideas from outside of Western philosophy into
technology ethics, another fascinating non‐Western philosophical tradition that
has recently been incorporated into technology ethics (at least a little bit!6) is the
ubuntu tradition from southern Africa. So, let us also briefly consider some of
the key ideas from this traditional way of thinking about ethics.
2.38 Ubuntu philosophy comes from traditional myths, folk wisdom, and
proverbs. The term “ubuntu” derives from a Nguni proverb “umuntu ngumuntu
ngabantu.” That proverb is often translated as saying that “a person is a person
through other persons” or “I am because we are.” As this proverb helps to
illustrate, it is a key aspect of ubuntu philosophy that community and social
relations are seen as having great value and importance. Indeed, social relations
are even viewed as being constitutive of what it is to be a human being and of
how we become human persons (“a person is a person through other persons”).
2.39 The idea of being human, or of being a human person, is an ideal or a
normative concept, rather than a purely descriptive concept within this way of
thinking. All human beings are viewed as having a potential to become more
fully human or, as we might put it, more humane. On some interpretations of
ubuntu philosophy, the most important ethical duty of each person is to try to
become fully human: in other words, to try to fully develop our human character
to the highest degree.
2.40 A fascinating aspect of this philosophy is that it views it as impossible to do
this on one's own. We help each other to become the best versions of ourselves
or, in this terminology, to become fully human. Therefore, we not only have a
duty to try to become better versions of ourselves—with more compassion for
others, more generosity toward others, and so on. We also have a duty to try to
help those around us to become better versions of themselves.
2.41 In this way, ubuntu philosophy can be interestingly compared and
contrasted with the well‐known idea from Rene Descartes according to which “I
think, therefore I am.” In a philosophical rendering of the above‐quoted ubuntu
proverb by the ubuntu scholar John S. Mbiti, we should not go with the
rationalistic and somewhat solipsistic or self‐centered idea that “I think,
therefore I am.” We should instead accept the more social‐relational idea that “I
am because we are; and since we are, therefore I am.”
2.42 As Mbiti sees things, then, we become who and what we are because of the
people we interact with, together with whom we go through life. The most
important aspect of a human person is not that they are able to think, in other
words, but that they have the potential to enter into mutually supportive
relationships with other people.7
2.43 Notably, there are some fascinating parallels (as well as differences, but let
us stick with the parallels) between the ethical ideas within ubuntu and
Confucian philosophy, on the one hand, and the ideas within ancient Greek
philosophy, such as the philosophy of Aristotle, on the other hand. The theme of
developing one's human potential to the fullest is a theme that runs through all of
these forms of ethical thought. A fascinating parallel between ubuntu philosophy
and Aristotle's ideas about friendship, moreover, is the idea that it is within
social relationships with other people that we stand the best chance of becoming
the best versions of ourselves.
2.44 In ubuntu philosophy, as just noted above, it is seen as impossible to
become fully human outside of interactions with other human beings who
mutually support each other. In Aristotle's philosophy, the best way to become a
virtuous person is within the context of so‐called virtue friendships within which
people help to bring out virtue in each other and develop together on the basis of
shared values.
2.45 In short, one key thing that the three forms of ethical philosophies we have
considered so far have in common is the conception of ethics as being about how
we can flourish and become the best versions of ourselves as human beings. It is
a fascinating question for technology ethics to ask whether technologies—
especially advanced modern technologies such as AI, robots, information and
communication technologies (e.g., like social media), tracking technologies, and
so on—are likely to help us become the best versions of ourselves. Or might
some modern technologies rather pose threats to our potential to become the best
versions of ourselves?
2.46 That is one of the key questions that those who incorporate these different
traditional ideas into technology ethics are beginning to ask. For now, though, let
us put that fascinating question aside, and stick with the more general question
of this chapter, namely, the question of what ethics is. Let us explore some
further perspectives.
2.4 Kantian Ethics
2.47 We just saw that in the traditional southern African ubuntu way of thinking
about ethics, we have a duty to try and become fully human and a duty to try and
help those around us to become fully human as well. This idea can be compared
and contrasted in an interesting way with the ethical principle from the
Enlightenment philosopher Immanuel Kant that was already mentioned in
Chapter 1—namely, Kant's formula of humanity. That is the ethical principle that
says that we should so act that we always treat the humanity in each person—in
our own person as well as in other people—as an end in itself, and never merely
as a means.
2.48 To be sure, there are many clashes and contrasts between the ethical
thinking of Kant, on the one hand, and the ethical thinking in a tradition like the
ubuntu tradition, on the other hand. But it is interesting to notice that at least on a
surface level, there seems to be a joint agreement that humanity should be
treated as an end, and that this applies both to our own humanity and the
humanity in others. The ubuntu tradition refers to a better version of ourselves
when it says that we have an ethical duty to try and become fully human and to
help those around us to be human. Similarly, Kant also uses a normatively
loaded conception of what humanity is when he says that we should treat the
humanity in each person as an end and never as a means only.
2.49 When Kant explains what his principle of persons as ends‐in‐themselves
means in practice, he makes a distinction between what this formula means for
how we should treat ourselves and what it means for how we should treat other
people. Kant thinks that we do not only have duties toward other people. We also
have duties toward ourselves.
2.50 In our own case, one of the main things Kant thinks that treating our own
humanity as an end‐in‐itself amounts to is that we should develop our capacities
and talents. Kant also thinks that treating our own humanity as an end‐in‐itself
means that we should never rob ourselves of our capacity to act on the basis of
moral principles. In other words, we should avoid things or situations that might
compromise or corrupt our ability to act on the basis of our moral convictions.
Being a person who has moral principles is an important source of self‐respect,
Kant thinks.
2.51 When it comes to other people, in contrast, treating their humanity as an
end‐in‐itself amounts to two main things. On the one hand, it involves including
other people's happiness among our ends (i.e., making it into one of our goals
that we try to make other people happy). On the other hand, it involves always
acting in ways that show respect for other people and their will. For example,
this means that we should avoid treating people in ways they do not or could not
consent to. And it means that we should avoid treating people in any way that
offends, degrades, or otherwise fails to honor their human dignity.
2.52 We can also briefly compare Kant to Confucius in light of what was said in
the section above. We noted there that the authors of the Analects attribute a
version of the Golden Rule to Confucius (“what you do not desire for yourself,
do not do to others”). Kant also develops what is often interpreted as a version or
elaboration of the Golden Rule. In Kant's case, the principle he formulates is
sometimes dubbed the universal law formula. This principle says the following:
“always act on maxims that you would be willing to lay down as universal
laws.”
2.53 A “maxim” refers to a personal rule or principle that we adopt for ourselves.
So, this universal law formula can also be formulated in the following way: act
on the basis of rules or personal principles that you would be willing to lay down
as universal laws for everyone. In a more Golden Rule‐like formulation: do not
act on the basis of any rules that you would not be willing to allow everyone else
to act on, but do act on the basis of rules that you would be willing to allow—or
perhaps even encourage—everyone else to act on.
2.54 As was also noted in chapter one, within Kant's ethical theory, anything that
is not a person with reason and a moral conscience is considered a “thing.” This
means that from Kant's perspective, animals are things. For Kant, this means that
strictly speaking, all animals can be treated as mere means to human ends, just
like the instrumental theory of technology regards all technologies as mere
means to human ends.
2.55 However, Kant thinks that there can be indirect moral duties toward
animals. Ultimately, these are duties toward ourselves and toward other human
beings. If we are cruel toward animals, Kant argues, that might harden us and
ultimately make us less able to treat human beings well, even to the point that we
might also become cruel toward human beings. This is an indirect moral reason
to avoid cruelty toward animals.
2.56 In the same way, some technology ethicists have argued that even if
technologies are always strictly speaking tools or mere means toward human
ends, it might be that there are certain technologies—e.g., robots or avatars that
look like human beings—that it would be best to avoid treating in a “cruel” way
so as to make sure that we do not become cruel toward human beings.
2.57 For example, some have worried that if people are impolite and bossy
toward voice assistant technologies—such as, for example, Amazon's “Alexa”
speaker or the “Siri” voice assistant on iPhones—then this might make them
start treating other human beings in impolite or bossy ways. Or if children see
their parents barking commands at voice assistant technologies in cruel ways,
then these children might get the idea that this is a normal way of requesting
others' assistance. To teach our children to be polite to other people, some people
have argued, it is best to not allow them nor to allow ourselves to act in
seemingly cruel or otherwise objectionable ways toward voice assistants or other
humanlike technologies.8 This argument is very similar to Kant's argument
regarding animals.
2.58 Similarly, when Kathleen Richardson argues that objectifying attitudes and
behavior toward sex robots might lead to objectifying attitudes and behaviors
toward human sex partners, that argument also has similarities with Kant's
argument regarding indirect duties toward animals. In Richardson's case, though,
she argues that the best solution is to ban humanoid sex robots. An alternative
“solution” would be for people to treat humanoid sex robots in apparently
respectful ways, so that they would train themselves to be respectful toward
human sex partners. That argument would be even more similar to Kant's
argument regarding indirect duties toward animals.
2.5 Utilitarianism and Consequentialist Ethical
Theories
2.59 A distinctive feature of Kant's ethical theory that we noted above is that it
only regards human beings with reason and a capacity to act on moral principles
as having the status of being moral persons, toward whom we can have direct
duties. On Kant's theory, since we ourselves are moral persons in this sense, we
have moral duties toward ourselves. And we also have moral duties toward other
human beings because they are moral persons with reason and a will that we
should respect. Any other being—such as an intelligent alien—that would also
have reason and be able to act on moral principles would also be a moral person
that we should show respect toward and treat as an end, as Kant sees things. But
animals, we have seen, are not moral persons in their own right. They lack
human‐like reason and the capacity to act on moral principles that we have.
2.60 Things are very different from the point of view of so‐called utilitarian
ethical theories. The founder of modern utilitarianism was an eccentric late 18th
century legal scholar, societal reformer, and philosopher named Jeremy
Bentham.9 He famously wrote: “ask not ‘Can they talk?’ or ‘Can they reason?’,
but ‘Can they suffer?’”10 This seems to be an allusion to an ancient Roman
ethics text by the statesman and philosopher Cicero. Cicero wrote that what
gives human beings a special dignity, which sets them apart from all other
beings, is that they can talk and reason.11 Bentham's just‐quoted line appears to
partly be a response to Cicero. But it could just as well have been written as a
response to Kant.
2.61 On Bentham's view, the only thing that matters in itself from a moral point
of view is the capacity for having pleasant or unpleasant experiences. Since
animals can experience pleasure and pain just as human beings can, Bentham
argued that it is possible to have direct moral duties toward non‐human animals,
just as we have direct moral duties toward human beings.
2.62 Within traditional so‐called hedonistic utilitarianism, we have one and only
one overarching moral duty: to promote the overall level of happiness in the
world, by bringing about pleasure and relieving pain and suffering. Sometimes
this is put in language that sounds like it was taken out of an economics book:
we should maximize pleasure and minimize pain, just like some economists talk
about maximizing profit and minimizing losses.
2.63 Actually, Bentham was the person who coined the term “maximize.” And
lots of early utilitarian thinkers were also economists—in addition to many other
things. This had to do with the fact that they had ambitions to reform the whole
of society, so that everything in life—from the law, to other social institutions, to
one's personal conduct—was guided by the principle of utility. Everything we do
and everything in society should strive to maximize pleasure and minimize pain.
No wonder that the early utilitarian philosophers, having such grand ambitions,
did not only discuss personal ethics but all other aspects of life as well.
2.64 To many people, this single‐minded focus on pleasure‐maximization and
pain‐minimization has seemed to be a too narrow conception of ethics. Surely
there is more to life than trying to maximize pleasure and minimize pain. It is
demeaning to human beings to think that the only thing worth pursuing is
pleasure and the only thing worth avoiding is pain. There are also other things
that are important in life. Or so critics of early forms of utilitarianism argued.
Notably, some of those critics were themselves utilitarian philosophers.
2.65 Like Bentham, John Stuart Mill was also a social reformer, philosopher,
economist, politician, and more. He was born while Bentham was still alive, and
was partly raised by Bentham, who was the best friend of his father, James Mill.
While his father was a convinced follower of Bentham's utilitarianism, John
Stuart Mill was among those who felt that early utilitarianism was too single‐
minded. But Mill did not want to abandon the basic utilitarian ethos altogether.
So, what Mill did was to argue that not all pleasures are equal, and that not all
forms of pain and suffering are equal.
2.66 According to Mill, some pleasures are better than others, and some forms of
suffering are worse than others. In particular, the intellectual pleasures of a
human being are more worth wanting than the simpler pleasures of non‐human
animals. In a famous passage, Mill writes:
It is better to be a human being dissatisfied than a pig satisfied; better to be
Socrates dissatisfied than a fool satisfied. And if the fool, or the pig, are of a
different opinion, it is because they only know their own side of the question.
12
(Quoted in Driver 2007: p. 49)
Mill is here making use of a type of argument that Plato developed in the
Republic, i.e., the dialogue which was mentioned above in the section about
Ancient Greek philosophy. That dialogue also features Socrates (just like Mill's
quote here does). Socrates is the main character in Plato's Republic. He argues
that when we compare different forms of life in terms of which ones are the best,
we should consult the opinions of those who have experience with the different
forms of life, and give less weight to the opinions of those who lack experience
with the different forms of life.
2.67 In the same way, Mill argues that when we compare different forms of
pleasure in order to ask which forms are “higher pleasures” and which are
“lower pleasures,” we should consult the opinions of those who have
experienced and learned to appreciate different forms of pleasure. The higher
pleasures, as Mill calls them, are types of pleasures that are consistently
preferred by those who have experienced different forms of pleasure.
2.68 According to Mill, mature human beings tend to consistently prefer forms
of pleasure or activities that make use of our intellectual, distinctively human
faculties. We attach a higher value to such pleasures, Mill thinks, than to simpler
forms of pleasure, when we are in a position to make well‐informed
comparisons. So just like Cicero and Kant assigned higher importance to human
beings than to non‐human animals, Mill assigned a higher value to human
happiness than to the happiness of animals.13
2.69 Mill's attempt to add more variety to utilitarian philosophy opened the door
to the development of even more pluralistic utilitarian conceptions of what is
good in life or, in other words, what should be promoted according to broadly
utilitarian ethical theories. These broader theories are often called
consequentialist theories. These days, most philosophers who are inspired by the
spirit of Bentham and Mill's utilitarianism typically accept the idea that it is our
most general moral duty to promote the overall good, but adopt pluralistic
theories about what goods there are in life.
2.70 For example, a consequentialist might say that things such as friendship,
love, knowledge, human achievement, justice, fairness, and different forms of
excellence are all among the good things in human life. Their consequentialism
in their ethical thinking consists in how they think that we should understand
ethics as being about a general duty to do whatever promotes the overall amount
of good in the world, so that we maximize the overall amount of whatever has
value in the universe. Different consequentialist ethical theorists can then differ
in what things they think are among the most important goods in life. And they
can also differ in their views about how to best go about promoting the goods
that they think we have ethical duties to promote.
2.71 This means that consequentialism, as such, is simply a general schema that
says “so act that you promote the overall good in the world.” This schema can be
filled out in many different ways, depending on what theories of the good we
adopt and depending on how one best promotes those goods.14 For example, one
consequentialist might say that the most important things in life are happiness
and knowledge, whereas another consequentialist might say that the most
important things in life are human achievement and justice. They would both be
consequentialists. But they would have very different ideas about what exactly
our more specific duties are. Or perhaps two different consequentialists agree
that happiness, knowledge, human achievement, and justice are the most
important goods in life, while disagreeing about how best to go about promoting
these goods.
2.72 Consequentialists also sometimes disagree among themselves about what
theories qualify as truly being within the broad class of consequentialist theories.
Mill, for example, thought that Aristotle was a utilitarian philosopher, since
Aristotle thinks that we should try to achieve eudaimonia, i.e., the particular
conception of happiness that Aristotle articulates within his ethical theory. Most
other consequentialist philosophers, however, prefer to distinguish a theory such
as Aristotle's from the types of consequentialist theories that they themselves
defend.15
2.73 For our purposes here, the exact details of such debates are not important.
What is more important to know is that even within particular schools of thought
within ethical theory, there can be lots of disagreement about what the best
versions of the ethical theories of the given schools are. This was the case back
in the ancient world, with the ethical theories developed within different schools
of thought then (e.g., Stoicism, Skepticism, Epicureanism, Cynicism, and so on).
And it is also the case within contemporary ethical theory.
2.74 In general, when philosophers discuss ethics, they tend to find that there are
actually many things they agree on. So, there is almost always at least some
common ground that can be used as a starting point of ethical discussions. But
philosophers and others who reflect on ethics also tend to find that there are
some things that are very hard to reach agreement on. This observation that
ethics involves many disagreements and many debates that are very hard to settle
is frustrating to some. To others, it is exhilarating.
2.6 If Ethics More Generally Can Be All the
Things Discussed in the Previous Sections, then
What Does this Mean for Technology Ethics in
Particular?
2.75 There are more perspectives on what ethics is about than those surveyed
above. For example, there are also so‐called social contract theories of ethics.
We will return to such contractualist theories in the next chapter. But for now,
these just‐considered perspectives provide a good overview of some of the main
theories there are about what ethics is all about. We can now put these ideas
together with the ideas from the previous chapter about what technology is. We
can thereby form an idea of what technology ethics is.
We have seen that:
Ethics concerns itself with, among other things, questions about what a good
life is/what it is to flourish as a human being. It concerns itself with our
human self‐understanding, with self‐cultivation and self‐improvement, and
with the role of different virtues and excellences in a well‐lived life. Ethics is
also about what ways of interacting with, or of relating to, those around us we
should favor or aim for; duties towards ourselves and duties towards others;
and duties related to the consequences of our actions. It explores theories of
what goods or values ought to be promoted, and about what grounds moral
status. And it investigates arguments regarding what should be permitted and
what should be forbidden.
In the previous chapter, we saw that:
Technologies can sometimes be viewed as tools or instruments that serve as
means to human ends, as parts of human activities. But technologies can also
be seen as things that shape how we perceive the world and what we are able
to do. And, moreover, some technologies can also potentially be seen as novel
forms of moral agents and moral patients, which are more than simply value‐
neutral tools.
We can therefore, somewhat loosely, offer the following definition of what
technology ethics is or is about:
Technology ethics concerns itself with questions about, among other things:
the relation(s) between technologies and what a good life is/what it is to
flourish as a human being;
the impact of technologies on our human self‐understanding, self‐cultivation,
and self‐improvement, including how technologies affect how we perceive
the world and ourselves and what we are able to do;
the role of different virtues and excellences in a well‐lived life and how they
might be affected by technologies;
what ways of interacting with those around us or of relating to those around
us we should favor or aim for, and how technologies might mediate these
relationships or potentially perform even more significant roles within such
relationships, such as perhaps becoming a new form of friends or romantic
partners;
duties towards ourselves and duties towards others, where these duties might
be carried out by us human beings or perhaps by certain novel forms of moral
agents in the shape of artificially intelligent technologies;
duties related to the consequences of our actions, which may depend on what
technologies we use;
theories of what goods or values ought to be promoted, and of what roles
technologies play in relation to the good;
theories about what grounds moral status, including the question of whether
any technologies might possess some form of moral status;
questions about what technologies should be permitted and what technologies
should be forbidden.
In other words, technology ethics is about many different things! It is not only
about the last bullet point about what should be forbidden and what should be
permitted.
2.76 As mentioned above, that last point is an important issue. It makes sense to
think long and hard about it. It is a very crucial part of what technology ethics is
all about. However, as was also noted earlier on, a conception of technology
ethics that primarily focuses on what should be permitted and what should be
forbidden would be a very narrow conception of technology ethics. It would also
be a much more boring conception than one that also concerns itself with
questions about the good life, virtues, human self‐understanding, novel forms of
moral agents and patients, and so on and so forth. In this book, the broader
conception of technology ethics that is outlined above will be our focus
throughout the rest of the discussion.
2.77 Also, technology ethics can also be about more things than those mentioned
in the definition offered above. The rough definition above is supposed to give a
perspective on what technology ethics can be. It is not supposed to offer a
perspective on all that technology ethics is or should be taken to be. Just as
technological developments often involve new innovations, technology ethics
may also sometimes need to involve theoretical innovations. Such developments
may be unpredictable and surprising.
2.78 In a way, technology ethics can be likened to the art of jazz music. A lot of
jazz is based on a set of standard compositions, which provide the basic structure
of the music and main melodies and motives. But then the role of the skilled jazz
musician is to creatively build on these basic structures and to improvise new
innovations, which help to extend the music that was already there to new
horizons. In the same way, technology ethics typically tends to build on the
foundations provided by traditional and more general ethical theory. However,
like jazz musicians, technology ethics researchers need to be creative. They
often need to improvise and try to extend—and to challenge—existent ideas and
theories from traditional ethical theory. The last thing we will do in this chapter
is to briefly reflect on a few ways in which this can happen.
2.7 How Technology Ethics Can Challenge and
Create a Need for Extensions of More General
Ethical Theory
2.79 One important potential problem with making use of traditional ethical
theories within technology ethics is that those traditional ethical theories
developed before there were any modern technologies around. They developed
before we had things like robots, artificial intelligence, social media, brain
implants, and other potentially morally confusing new technologies in society. In
short, the theories developed with human‐human interaction in mind, not with
human‐technology interaction in mind. So, there is no guarantee that the ideas
developed for the ethics of human‐human interaction during the long history of
ethics are going to always naturally carry over to the modern world of ever‐
increasing human‐technology interaction.
2.80 It is also like this when it comes to the law. For example, as the legal
scholar Jacob Turner has noted, laws related to driving were developed before
there was any such a thing as a self‐driving car. Therefore, these laws related to
driving all assume that there is always a human driver who can potentially be
held responsible if there is any problem, and who has so‐called duties of care to
make sure that there are as few accidents as possible. Now we have the new idea
of a car that drives itself. So, laws that assume that there is a human driver will
not always easily carry over to the new context of self‐driving cars. The cars
themselves might not have moral or legal responsibilities, and also no duties of
care, at least not in the ways that human beings do. So, there is a need for new
laws, or so Turner argues.16
2.81 In the same way, our traditional moral theories all assume that the main
moral agents and moral patients are human beings (and animals). So, there might
be a need for new moral principles and new moral ideas, at least in relation to
some modern technologies that were not in existence when our traditional moral
theories were formulated by the people and societies discussed above.
2.82 For example, if robots or other forms of artificial intelligence need to make
morally sensitive decisions, can they and should they follow the advice offered
by the types of moral theories that have been considered above, which were
developed with human beings in mind? There is no guarantee that ethical
theories that were developed to guide human decision‐making and human action
will easily carry over to the new case of decision‐making and actions on the part
of advanced technologies. More on this in Chapter 7.
2.83 Another worry about whether traditional ethics is a good fit for the modern
world comes from the philosophers Ingmar Persson and Julian Savulescu. They
argue that a lot of our human moral attitudes developed during the long period of
human evolution, where human beings lived in small tribes and only made use of
very simple technologies. The human mind— and our human moral conscience
—was not developed to deal with the complexity of the modern world or the
powerful nature of our modern technologies, Persson and Savulescu argue. But
now we live in the modern world, not in small tribes with simple technologies.
We live in huge modern nations with extremely powerful technologies, with
which we are quickly using up the world's natural resources and creating risks of
great harm to the planet and the future of humanity.
2.84 Our human psychology, including our moral attitudes that are part of our
human psychology, are not well‐adapted to this modern world that we are
finding ourselves in, according to Persson and Savulescu. So, we might not only
need to update our moral theories and ideas, they argue; we might also need to
update ourselves. We need, they argue, some form of “moral enhancement” of
our human nature, so as to enable us to deal with the challenges of the
contemporary world and the technologies we have created.
2.85 For example, the average person does not feel a sufficiently strong sense of
responsibility for the part they play in causing human‐made climate change. And
so, the average person just continues on as usual, even if he or she constantly
hears on the news that the world is facing a devastating climate catastrophe
because of how we are living our lives in the modern world. Persson and
Savulescu view this as an example of how our human psychology—along with
our everyday moral thinking—is unfit for the modern world. We are, and our
moral thinking is, “unfit for the future,” they argue.17
2.86 Other, also dramatically‐minded authors are arguing that artificial
intelligence is potentially going to spiral out of control, and that our current ways
of thinking about ethical duties and responsibilities are not equipped to allow us
to deal with the possible impact of powerful artificial intelligence on human
society. According to this way of thinking, technologies like artificial
intelligence, robots, and future technologies we cannot yet imagine may require
us to develop new ideas and theories about what ethical duties and
responsibilities we as human beings have. Some people even worry that
technologies like artificial intelligence might pose an “existential risk” to human
beings, just like climate change does. So, we do not only need new laws to deal
with these new risks. We may also need new and fresh ideas about our ethical
duties and responsibilities.18
2.87 None of this is to say that traditional ethical theories cannot and should not
be part of the solution to the new problems within technology ethics. It is surely
too extreme and unrealistic to suggest that we need to altogether abandon our old
ideas about ethics, our widely shared moral intuitions that we have as human
beings, or our traditional ways of understanding ourselves and our place in the
world. However, technology ethics does put pressure on the ways in which we
have traditionally thought about ethics. It also puts pressure on our traditional
ways of thinking about ourselves and our role in the world as human beings.
This is part of what is exciting about technology ethics.
Annotated Bibliography
Julia Driver (2006). Ethics, the Fundamentals, Oxford: Wiley‐Blackwell.
An accessible introduction to ethics, which, among other things, provides a
good overview over ethical theories like utilitarianism, Kantian ethics, virtue
ethics, and social contract theory.
Fainos Mangena (2016). “Hunhu/Ubuntu in the Traditional Thought of Southern
Africa,” Internet Encyclopedia of Philosophy: https://iep.utm.edu/hunhu.
An overview of southern African ubuntu ethics.
Ingmar Persson and Julian Savulescu (2012). Unfit for the Future: The Need for
Moral Enhancement, Oxford: Oxford University Press.
A provocative book that argues that our human moral sensibilities may not be
well‐suited to deal with the ethical problems and global challenges associated
with life in the modern world.
Kathleen Richardson (2015). “The Asymmetrical ‘Relationship’: Parallels
between Prostitution and the Development of Sex Robots,” SIGCAS
Computers & Society, vol. 45 no. 3, pp. 290–293.
Presents the main arguments for why the campaign against sex robots thinks
that such robots should be banned.
Bernd Carsten Stahl (2021). Artificial Intelligence for a Better Future, Berlin:
Springer.
An accessible overview of ethical issues related to AI, which argues that a
virtue‐ethical perspective focused on human flourishing should serve as the
basis of AI ethics.
Jussi Suikkanen (2014). This is Ethics: An Introduction, Oxford: Wiley‐
Blackwell.
A comprehensive and accessible introduction to ethics. Also presents a broad
conception of what ethics is about.
Jacob Turner (2019). Robot Rules: Regulating Artificial Intelligence, Berlin:
Springer.
A legal expert explains how technologies like self‐driving cars and other
forms of AI may require us to rethink many current legal ideas. Highly
relevant for those interested in how technologies might also require us to
rethink some of our traditional ideas about ethics.
Pak‐Hang Wong and Tom Xiaowei Wang (2021). Harmonious Technology: A
Confucian Ethics of Technology, London: Routledge.
A collection of essays exploring how issues in technology ethics can be
approached from the point of view of Confucian ethics.
Notes
1 All the just‐made quotations are from the above‐cited website. Here, in
addition, is a video that introduces the campaign: “The Stop Killer Robots
Campaign”: https://www.youtube.com/watch?v=‐PVmPbFJm9E
2 See this video: “Why We're Changing Our Name at the Campaign Against Sex
Robots”: https://www.youtube.com/watch?v=QkgbTYHVdrA. For a debate
between Richardson and the computer scientist Kate Devlin, who takes a
rather different view, see the video “Sex, Robots, and Artificial Intimacy”:
https://www.youtube.com/watch?v=kDC82yhpZ7A
3 You can learn more about Plato's Republic in this episode of “In Our Time”:
“Plato's Republic”: www.bbc.co.uk/sounds/play/b08vwn6h
4 Peter Adamson's podcast “The History of Philosophy Without Any Gaps”
presents the ancient philosophers' ideas about virtue and the good life in a
clear and engaging way. See particularly the episodes on philosophy in the
classical Greek philosophy: https://historyofphilosophy.net/series/classical‐
greek‐philosophy and Hellenistic philosophy:
https://historyofphilosophy.net/later‐antiquity/hellenistic‐philosophy
5 For a short video on Confucius, see “Who Was Confucius?”, by Bryan W. Van
Norden: https://www.youtube.com/watch?v=wFt_VGG0kJU
6 It says “at least a little bit!” above since there is not yet many examples of this.
Here is one example, from Sabelo Mhlambi (2020) at Harvard University's
Carr Center: “From Rationality to Relationality: Ubuntu as an Ethical and
Human Rights Framework for Artificial Intelligence Governance”:
https://carrcenter.hks.harvard.edu/files/cchr/files/ccdp_2020‐
009_sabelo_b.pdf. In Chapter 8 we will consider more examples of how
ubuntu ethics can be used within technology ethics, when we discuss whether
machines can be moral patients.
7 For more on this idea, see “Descartes Was Wrong: ‘A Person is a Person
Through Other Persons',” by Abeba Birhane (2017), in Aeon, here:
https://aeon.co/ideas/descartes‐was‐wrong‐a‐person‐is‐a‐person‐through‐
other‐persons
8 Check out this video in which the philosopher Olya Kudina talks about the
topic: “Hey Siri, Why Are My Kids Screaming at You?”:
https://www.youtube.com/watch?v=ve6qJGt1_kk
9 Here is an episode of the great “Philosophy Bites” podcast in which Philip
Schofield provides more background on Bentham: “Philip Schofield on
Jeremy Bentham's Utilitarianism”:
https://philosophybites.com/2012/02/philip‐schofield‐on‐jeremy‐benthams‐
utilitarianism.html
10 That quote is taken from Bentham's (1789) An Introduction to the Principles
of Morals and Legislation, which can be read online here:
https://oll.libertyfund.org/title/bentham‐an‐introduction‐to‐the‐principles‐of‐
morals‐and‐legislation
11 In book 1 of his De Legibus (On the Laws), Cicero suggests that our ability to
speak, think, and reason raises human beings to above animals to an almost
divine level. For more information, see, for instance, the Wikipedia entry on
“De Legibus”: https://en.wikipedia.org/wiki/De_Legibus
12 The book by Mill (1863) from which this quote is taken—a book simply
called Utilitarianism—can be read online, here:
https://www.gutenberg.org/ebooks/11224
13 Roger Crisp discusses Mill's utilitarianism on this episode of the “Philosophy
Bites” podcast: “Roger Crisp on Utilitarianism”: https://philosophybites.com/
2007/07/roger‐crisp‐on‐.html
14 Some consequentialists are interested in what is called “effective altruism.”
You can listen to Beth Barnes on that idea here: “Effective Altruism|Beth
Barnes|TEXxExeter”: https://www.youtube.com/watch?v=LtWINl3C_7s
15 You can hear Philip Pettit discuss consequentialist ethics on the “Philosophy
Bites” podcast here: “Philip Pettit on Consequentialism”:
https://philosophybites.com/2011/09/philip‐pettit‐on‐consequentialism‐1.html
16 Here is an episode of the “Dolores Project” podcast with Turner as the guest:
“AI, Robots, and the Challenge of Policy Making w/Jacob Turner”:
https://anchor.fm/joshua‐k‐smith6/episodes/AI‐‐Robots‐‐and‐the‐Challenge‐
of‐Policy‐Making‐ w‐Jacob‐Turner‐e189ppi
17 You can hear Savulescu speak about these ideas in this “Virtual Philosopher”
podcast episode: “Julian Savulescu on Moral Enhancement”:
https://virtualphilosopher.com/2011/05/julian‐savulescu‐on‐moral‐
enhancement‐.html
18 You can listen to Toby Ord speak about the ethics of existential risks on the
“80,000 Hours” podcast here: “#72: Toby Ord on the Precipe and Humanity's
Potential Futures”: https://80000hours.org/podcast/episodes/toby‐ord‐the‐
precipice‐ existential‐risk‐future‐humanity
3
METHODS OF TECHNOLOGY ETHICS: THE
ETHICS OF SELF‐DRIVING CARS AS A CASE
STUDY
3.1 Methodologies of Ethics?
3.1 When academic researchers apply for research grants, they typically have to
complete various forms. The application forms usually have a section in which
the researchers need to describe what methods they use. This methodology
section is one that philosophy researchers, including philosophers focusing on
ethics, often find it a bit hard to fill out. This is not because there are no methods
that they are using. (To paraphrase a line from Shakespeare's Hamlet, there are
certainly “methods to the madness” that is philosophy!) It is rather because the
choice of method is often itself a philosophical—or indeed even an ethical—
issue that needs to be debated and reflected on. Philosophers tend to disagree
about what methods are the best ones to use. When it comes to ethics, it can
sometimes even happen that certain methods of studying the topic may appear to
be ethically questionable to some. More on this later.
3.2 So how do philosophers typically deal with the methodology section in grant
applications? What many philosophers end up doing is to say that they will be
using the so‐called reflective equilibrium methodology. The idea is that in
thinking about philosophical issues, one should continually be moving back and
forth between more general ideas, principles, and intuitions, on the one hand,
and more specific ideas, judgments, and intuitions, on the other hand. One then
tries to make informed and well‐considered judgments about the more general
ideas with an eye to how they relate to the more specific ideas. And one also
tries to make informed and well‐considered judgments about the more specific
ideas with an eye to how they relate to the more general ideas.
3.3 Eventually, the hope is that one reaches a “reflective equilibrium,” whereby
one has adjusted one's more general ideas so that they fit better with one's more
specific convictions, and one has also adjusted one's more specific ideas so that
they fit better with one's more general convictions. The overall package of ideas
is supposed to form a coherent whole. The more general ideas should help to
explain the more specific ideas. And the more specific ideas should help to
illustrate or confirm the more general ideas. If this happens, internal coherence
among the ideas has been achieved. And the more general and more specific
ideas will stand in a mutually supportive relationship to one another. Most
philosophers agree that this is part of what one wants to achieve on the basis of
one's philosophizing about some topic.
3.4 This reflective equilibrium method is one of the most general methods that
philosophers tend to use when they think about ethics and other topics. So no
wonder that many philosophers write in their grant applications that they are
planning to use the reflective equilibrium method if they are awarded the grant
they are applying for! At the same time, many philosophers tend to wish that
they could say something more original and more specific than that they are
planning to use the reflective equilibrium method. After all, if pretty much all
philosophers use that very general method, there is nothing very distinctive
about declaring that one is planning to use it.
3.5 So, the question arises whether it is possible to say more about what methods
one can use. Is it possible to formulate, and to then highlight merits and
drawbacks of, more specific methods one might use when one is approaching
some philosophical issue, such as the issues that arise within technology ethics?
This is the topic this chapter concerns itself with. In other words, this chapter
discusses the question of what different, more specific methodologies we can use
in technology ethics. We will also consider some strengths and weaknesses of
those different methods.
3.6 It is possible to discuss this on a very general level. However, that quickly
gets pretty boring. Moreover, discussions about methodology on a very general
level can also become a little bit confusing. Things can become overly abstract
and dry. So it is better to look at a particular topic, and explore what methods can
be used in research about the topic in question. That way, things become more
interesting, more concrete, and, in effect, easier to understand. Accordingly, that
is the approach that this chapter takes.
3.7 The case study the chapter will focus on in order to illustrate different
methods of technology ethics is the ethics of self‐driving cars. More specifically,
the chapter will focus on the question of what self‐driving cars should do in
dangerous accident scenarios, where a crash is unavoidable, and this involves
moral dilemmas where different people's lives are at stake.
3.8 This topic has received a lot of attention during the last few years. One
noteworthy thing about this discussion is that almost every imaginable method
that can be used within technology ethics has been used to tackle this issue. So,
the case of the ethics of crashes involving self‐driving cars is a very useful case
to look at for anybody who wishes to survey different methods that can be used
in technology ethics.
3.9 The chapter will first offer some background about the ethics of self‐driving
cars. After that, it will discuss a number of different methods that have been used
to investigate this issue. Toward the end of the chapter, there will be some brief
discussion about whether we should be pluralists about methodology within
technology ethics (i.e., whether we should make use of multiple methods) or
whether we should be monists (i.e., whether we should favor one single method
only). Even though the chapter will highlight shortcomings (in addition to
strengths!) of the methods discussed below, the conclusion of the chapter will be
that all of these methods have a certain amount of value. For this reason, the
chapter will end by endorsing a pluralistic stance regarding methodology in
technology ethics.
3.2 The Ethics of Self‐Driving Cars
3.10 The expression “self‐driving car” will here be used to refer to automated
cars that can operate in a so‐called autonomous mode either in all traffic
scenarios or in at least some traffic scenarios. The technical term “autonomy”
refers to the capacity that a machine might have to operate on its own, at least
for some periods of time, performing certain tasks, without direct human
intervention.
3.11 3.12 Currently, no cars are available on the market that are fully automated
and that always operate in a functionally autonomous mode. But at the time of
writing, there are already cars on the market that are able to operate in a
functionally autonomous mode in at least some traffic situations. In the kinds of
cars that are currently available, the humans in the car always have to be ready to
take over control of the driving. But the cars can also drive themselves, at least
in some situations, such as on the highway. The “autopilot” function in Tesla's
Model S cars is an example of that kind of car. In what follows, all discussion of
the ethics of self‐driving cars will focus on those occasions when the cars are
operating in the autonomous mode and the humans in the car are not performing
any driving tasks.
3.13 Notably, the ethics of self‐driving cars is a very practical and concrete
example of the ethics of artificial intelligence. As mentioned in chapter one,
artificial intelligence is, on the most general level, defined as the capacity that a
machine or other technology might have to perform certain tasks that human
beings need their intelligence for. Since we need to use our intelligence when we
drive cars, a car that can drive itself in certain traffic situations possesses a
degree of artificial intelligence. That's why the ethics of self‐driving cars is a
very concrete example of the ethics of artificial intelligence.
3.14 When philosophers started writing about the ethics of self‐driving cars
around 2014, the discussion was based on hypothetical thought experiments. For
example, what if a self‐driving car were to get into a risky accident scenario?
What should the car do? What if a self‐driving car would injure or kill a human
being? Who should then be held responsible?1
3.15 Philosophers were imagining scenarios in which these sorts of things would
happen, in order to start thinking about these issues before they actually started
happening in real life, so as to be prepared for ethical questions that might soon
become real‐life practical issues. Legal scholars had been discussing these issues
a little bit earlier—already around 2012 or so. But those early legal discussions
also primarily revolved around hypothetical scenarios and thought experiments.
3.16 Shortly thereafter, however, what had previously been events imagined in
hypothetical thought experiments started happening in real life. In 2015, for
example, there were about 20 minor accidents involving self‐driving cars. What
happened in most of those cases was that people driving regular cars accidentally
drove into experimental self‐driving cars. The self‐driving cars behaved
differently than human‐driven cars—for example, by accelerating more slowly
and gently—which led some human drivers to rear‐end them. In 2016, however,
there was the first accident involving a self‐driving car that clearly was caused
by the self‐driving car. An experimental car operated by the company Google
drove into a bus, as the bus was coming in for a stop. In the accident report
afterwards, Google admitted that their car had caused the accident. Just as
nobody had been injured in any of the minor crashes in 2015, no human being
was injured in this crash either. There were only some minor scratches on the car
and on the side of the bus.
3.17 A little later in 2016, however, something more tragic happened. The first
person died while riding in a self‐driving car. In May of that year, a Florida man
was killed while riding in a Tesla Model S car that was operating in the
“autopilot” mode. The artificial intelligence in the car failed to recognize a large
white truck that appeared on the road as being a dangerous obstacle. The car
drove straight into the truck, with the result that the man in the self‐driving car
immediately died. The car continued driving for a little while only to then crash
into a tree.
3.18 In March of 2018, in turn, the first pedestrian was hit and killed by a self‐
driving car. An experimental self‐driving car operated by the ride‐hailing service
company Uber drove straight into Elaine Herzberg, as she was crossing the road
while walking with a bicycle. The artificial intelligence in the car first classified
Herzberg as an unknown object, then as a bicycle, and then as a vehicle. It kept
jumping back and forth between different classifications, and no appropriate
action was taken in time. The human safety driver in the car, who was supposed
to pay attention and hit the brakes if necessary, was also not paying sufficient
attention, and thus failed to spot Herzberg in time. Tragically, Herzberg died in
the ambulance on the way to the hospital.2
3.19 What had been thought experiments within philosophy and legal theory just
a few years earlier, in other words, had quickly become real‐life issues. The
lesson is this. Self‐driving cars will sometimes face accident scenarios in which
they have to be able to react in the correct way, and in which they might harm
human beings based on how they behave in traffic.
3.20 The main ethical issues that have been discussed in the ethics of self‐
driving cars—both in relation to imagined scenarios and in relation to the sorts
of real‐life cases just mentioned—are the following two questions: (1) how
should self‐driving cars be made to behave in dangerous scenarios, where
different people's safety is at risk, and it might not be possible to avoid harming
some of those people? And (2) who should be held responsible when a self‐
driving car operating in an autonomous mode does harm somebody?
3.21 Other ethical questions have also been discussed. For example, what level
of safety should we aim for and what risk‐levels can we tolerate in self‐driving
cars? If self‐driving cars eventually become much safer than regular cars, should
we still allow people to drive regular cars if they want to do so? Or should we
then only use self‐driving cars? Various other ethical questions can also be
raised. But as noted above, most of the existing ethical discussion has been about
the two just‐mentioned questions.
3.22 In illustrating different methods that can be used in technology ethics in this
chapter, we will focus on the above‐described first question. That is, we will
focus on the question of how self‐driving cars should be made to behave in risky
scenarios that might involve moral dilemmas where different people's safety is at
stake. There is a remarkable variety in the methods that have been used by those
who have concerned themselves with this issue.
3.3 Ethics by Committee
3.23 When it comes to AI ethics more generally as well as when it comes to the
ethics of self‐driving cars in particular, one approach that has become very
common is to assemble a team of experts who are supposed to come up with a
set of ethical guidelines. This might, as noted, be ethical guidelines for artificial
intelligence more generally, or ethical guidelines for a particular kind of AI—
such as self‐driving cars. For example, the European commission famously put
together what was called a “high‐level expert group” on artificial intelligence,
which was tasked with coming up with a set of ethical guidelines for AI. And—
to take another example—a team of experts in Germany put together the first set
of national ethical guidelines regarding self‐driving cars in 2017.3
3.24 The experts involved in these kinds of groups might come from different
fields, so that different stakeholders are involved and a broad range of
perspectives is represented. This is one potentially positive aspect of this
approach to a topic like the ethics of self‐driving cars. Another potentially
positive aspect of this approach is that a set of ethics guidelines produced by a
team of experts can appear to have a certain authority—and be worth taking
seriously—because it represents the consensus of a set of experts. However, this
method of doing ethics—viz. having a collected group of experts draft a set of
ethics guidelines—has also been criticized, and can be seen as having some
significant weaknesses.
3.25 One worry can be related to potential conflicts of interest. For example,
some—in fact many!—of the AI experts who were asked to join the so‐called
high‐level expert group mentioned above came from the technology industry.
And remarkably few members of this expert group were actually ethics experts.
One of those ethics experts—the German philosophy professor Thomas
Metzinger—has himself been very critical of this particular expert group and the
ethics guidelines that the group produced.
3.26 According to Metzinger, many of the ethical suggestions that he put
forward were ignored by the industry people involved. The industry people had
an interest, Metzinger argued, in coming up with vague ethics goals and in
avoiding ethics‐based regulation. The industry people also had an interest in
appearing to care about ethics, all while putting out diffuse and regulation‐
avoiding ethics guidelines, according to Metzinger's critique. This was a case of
“ethics washing,” Metzinger concluded. In general, while it can seem to make
sense to not only have ethics experts but also representatives from the tech
industry when one is putting together ethics guidelines about some form of
technology—e.g., AI—there is a big risk that ethics washing will happen.4
3.27 Moreover, while it can be a strength of one of these ethics guidelines
documents that it is a compromise of the views of a team of experts, the fact that
it is a form of compromise of what people on the committee of experts were
willing to agree with might also mean that it does not really represent anybody's
point of view. Or it can mean that the values or principles need to be formulated
in such an open‐ended way that they either lack “teeth” (as one criticism of such
documents has put it) or that they fail to provide much guidance by being open
to many different interpretations.
3.28 It has also been argued that some of these ethics documents that teams of
ethics experts have come up with have been too focused on certain ethical
perspectives while ignoring other perspectives. One criticism of the many AI
ethics guidelines that have been published in the last few years, for example, has
been that they have failed to include or take seriously the ethical perspectives of
non‐Western traditions and ways of thinking. They have, according to these
critical responses, been too Euro‐centric or too US‐centric, while modern
technologies have a global impact and ethics documents should therefore also
take into account non‐Western ethical perspectives. This is certainly a very
important criticism.
3.29 Here too, however, a worry might be that if all possible perspectives are
taken into account and some sort of compromise guidelines are created that take
all possible perspectives into account, then the resulting guidelines might
necessarily need to be very open‐ended and potentially again toothless. How
else, it might be worried, can one get people from all possible perspectives to
agree on a joint set of guidelines?
3.4 Ethics by Analogy: The Trolley Problem
Comparison
3.30 When philosophers and others started becoming interested in the ethics of
self‐driving cars around 2014, one of the very first things that they did was to
look around for previously discussed topics within ethics research that were
similar to the new topic of crashes involving self‐driving cars. In particular,
various people—philosophers, journalists, behavioral economists, psychologists,
and so on—quickly started comparing the ethics of crashes involving self‐
driving cars to the so‐called trolley problem from philosophy. It can certainly be
fun to compare a new ethical topic with something that has been discussed
before. It can also be instructive to do so, because what has been discussed in the
literature about the older topic might be brought to bear on the new topic.
3.31 The trolley problem stems from a set of examples developed by the
philosophers Philippa Foot, Judith Jarvis Thomson, and Frances Kamm—among
others—who imagine different scenarios in which saving the lives of a group of
people (e.g., five people) would involve harming or killing a small number of
people (e.g., one person). In the most well‐known example, an out‐of‐control
trolley is heading down some train tracks in the direction of five people who will
be hit and killed if the trolley continues going straight. However, there is a
switch with which the trolley can be redirected to a side track. You are standing
next to the switch. If you pull the switch and redirect the trolley, you will save
the five. But there is one person on the side‐track, and that person will be hit and
killed by the trolley if you pull the switch. What should you do in this case?
3.32 In another variation, there is no switch and no side‐track, but there is a
bridge over the train tracks. And there is a large and heavy person standing on
that bridge. You are standing behind that person. And you realize that if you
push that large person down onto the tracks, the trolley will hit and kill the large
person. But when it does so, the automated breaks would be set off, and the five
people on the tracks would be saved. What should you do in this version of the
case? Many people think that it would be okay to save the five in the first case
by redirecting the train using the switch, but that it would not be okay to save the
five in the second case by pushing the large person toward his death.
3.33 The expression “the trolley problem” has sometimes been used to refer to
the puzzle of explaining and justifying why it might be right to save the five by
doing something that would kill one person in some sorts of cases (e.g., the
switch case) but not in other cases (e.g., the bridge case). Actually, some of the
examples discussed in the trolley problem debate do not even mention any
trolleys. In one grim example that is sometimes discussed, for instance, a doctor
is reflecting on whether to kill one healthy patient so as to be able to transplant
their organs to five other patients in need of organ transplants. That case is often
discussed in relation to the trolley problem, but does not mention any trolleys.
The main issue in all these cases, according to Frances Kamm, is that we seem to
have different intuitions about different ways of saving many by killing or letting
few people die. This is something that is puzzling and that we need to reflect on
and try to explain or justify.5
3.34 We can easily come up with cases in which self‐driving cars are in
dilemmas where the only way to save five people is to have the self‐driving cars
drive into and kill one person. Accordingly, the ethics of crashing self‐driving
cars can perhaps be seen as a sort of real‐life trolley dilemma. So we can perhaps
learn something about the ethics of self‐driving cars by studying the trolley
problem and the literature about it. Some philosophers—e.g., Geoff Keeling,
Lucie White, and Dietmar Hübner—have argued that this is a good and
important way of approaching the ethics of self‐driving cars. However, there are
some reasons to be careful about drawing too close of an analogy between
something like the philosophy of the trolley problem and the real‐world ethics of
self‐driving cars. Here are three reasons why one might be careful—or perhaps
even skeptical—about this approach.
3.35 One reason to be careful about relying on comparisons with previous
philosophical discussions (such as the one about the trolley problem) when
approaching new technology ethics issues (such as self‐driving cars) is that
philosophical discussions often involve highly idealized thought experiments
that may self‐consciously try to avoid the messiness and complexity of real‐life
ethical issues. The trolley problem thought experiments, for example, ask us to
focus on a small set of imagined facts, to set all other possible concerns aside,
and to then have intuitions about these different cases. The role of the
philosopher who is philosophizing about the trolley problem—or of the
psychologist who is doing empirical studies about it—is to try to find interesting
patterns in people's moral intuitions. For that purpose, simplifying and
abstracting away many of the details, the complexity, or messiness of real life
can make a lot of sense. However, in technology ethics—especially when it is
about real‐life issues such as how self‐driving cars should behave on public
roads—it is often very important to try to take into account the full complexity
of real life.
3.36 A second reason to be careful about relying on something like previous
discussions about the trolley problem when thinking about a new technology
ethics issue like the ethics of self‐driving car is related to the first. It is that while
philosophical discussions about topics like the trolley problem often set aside
issues related to moral and legal responsibility, discussions about something like
the ethics of self‐driving cars cannot do so. When philosophy teachers use the
trolley problem in a classroom setting, a clever student often rightly asks:
“would one not go to prison if one pushed a large person to his death to save five
people, or even if one redirected a train to save five, knowing that one person
will die?” That is a question to which the philosophy teacher usually responds:
“yes, that might happen, but we are here engaging in a philosophical thought
experiment, and we can set such issues aside.” In the real‐world ethics of self‐
driving cars and in most other parts of technology ethics, we cannot set issues
about legal and moral responsibility aside. In fact, they are among the most
important and dominant topics being discussed in this field.
3.37 A third reason to be careful about drawing very strong comparisons
between classic philosophical cases like the trolley problem and technology
ethics issues like that of self‐driving cars has to do with risk and uncertainty. In
philosophical thought experiments like the trolley problem—and many others as
well—we often assume for the sake of the discussion that we know all the
relevant facts and that we know what exact outcomes different possible courses
of action would produce. When we deal with real‐world technology ethics, in
contrast, we usually have to think in terms of possible risks and possible
opportunities related to the technologies we are focusing on. And we are often
plagued with a lot of uncertainty about how likely different possible outcomes
are.
3.38 In short, it can be fun and interesting—and sometimes also highly
instructive— to make comparisons between new technology ethics issues and
older philosophical discussions, such as the one about the trolley problem. But
one also needs to be careful. There can be relevant differences that might
mislead our thinking in technology ethics if we are not careful enough. The three
issues mentioned above are examples of such reasons for caution.
3.5 Empirical Ethics
3.39 Staying on the topic of thought experiments a little longer, we can next
consider another interesting way in which the topic of crashes involving self‐
driving cars has been approached. We can consider what is called the “empirical
ethics” method. This approach has been taken by, among others, Jean‐François
Bonnefon and colleagues—a group of psychologists and behavioral economists
based at MIT. They were inspired by the philosopher‐turned‐psychologist Joshua
Greene's slightly earlier work. (As it happens, Greene's lab is located just across
the street from MIT, over at Harvard University.)
3.40 In his research, Greene has combined psychological studies of people's
reactions to moral dilemmas with philosophical/ethical premises, so as to
generate empirically informed ethical arguments. Some of that research makes
use of variations of the above‐described trolley cases.6 Similarly, Bonnefon and
colleagues also make use of vignettes inspired by the trolley problem in their
research. We will get to that in just a moment. But let us first consider another
interesting finding from their lab.
3.41 One widely publicized finding from Bonnefon et al. is the following. When
people are asked about how other people's self‐driving cars should be
programmed to handle accident scenarios, they are inclined to want other
people's cars to simply minimize overall harm. However, when surveyed about
what kind of self‐driving cars they themselves want to use, people tend to favor
cars that would save them in an accident scenario, even if this would not
minimize overall harm. At the very least, they do not want to be forced to buy
cars that would be “altruistic” by seeking to minimize overall harm. This was a
finding that received a lot of attention when it was first publicized in 2016. The
researchers argued that this asymmetry in people's attitudes must be taken into
account when we reflect on how self‐driving cars should be programmed to deal
with risky scenarios.
3.42 On an intriguing website set up by the same researchers—the “moral
machine” website7—one can explore various ethical dilemmas involving self‐
driving cars, illustrated by cartoon‐like drawings. This involves things such as
cases where the car can either go left and run over three grandmothers, or go
right and hit two children and three cats or dogs. Users of the website are asked
to give their opinions about what the car should do in a wide range of variations
on these kinds of themes, which are modeled on examples from the
philosophical trolley problem discussion. Millions of people world‐wide have
accessed the website and volunteered their opinions about what the cars should
do in the dilemmas depicted.
3.43 The researchers have taken these responses and identified patterns in
people's responses. They have found some interesting patterns, for example that
people in different parts of the world seem to have different attitudes about
whether it is more important to spare the lives of old people or young people.
Other findings include that depending on where one is in the world, one is likely
to judge people more or less harshly if they are jaywalking or crossing the street
when there is a red light. In some of the dilemmas, the question is, for example,
whether a car should run into a person illegally crossing the street on a red light
so as to avoid a crash where somebody in a self‐driving car would die.8
3.44 These different types of findings are certainly fascinating from a
sociological or anthropological point of view. But how well do these kinds of
findings work as normative premises in ethical arguments about how
technologies should behave? Should we rely on these kinds of findings in ethical
arguments? Here are three possible concerns that can be raised.
3.45 First, most people still have very little real experience with technologies
like self‐driving cars. It is likely that their attitudes might change once they have
more actual experience with these kinds of technologies. This suggests that we
should not put too much weight on people's current attitudes in ethical
arguments, though they are of course very interesting to be aware of.
3.46 Second, when people are asked to offer intuitive responses to an imagined
moral dilemma presented in a cartoonlike manner, they are typically not asked to
justify their responses. Instead, researchers are simply testing what gut reactions
or preferences research subjects have in response to the choices presented to
them. But in serious ethical discussions, it is important to carefully articulate and
assess arguments or reasons in favor of or against the different options that are
being considered.
3.47 Third, people appear to have inconsistent or paradoxical attitudes. In the
first finding described above, for example, many people want others to have
harm‐minimizing cars, while themselves wanting to have cars that would favor
them. It is of course very interesting to know that people's attitudes exhibit these
asymmetries. But we could not plausibly put forward an ethical argument
according to which some people should have harm‐minimizing self‐driving cars
(everyone else!) while others should have cars saving their owners (us!).
3.48 This clashes with the sort of golden rule approach to ethics that most people
will upon reflection agree with: if you want something for yourself, you should
out of ethical consistency not deny that thing to others. Alternatively put,
wanting different rules for others and for yourselves is usually considered to
clash with ethical thinking. People's apparently asymmetrical intuitive
preferences about self‐driving cars seem to clash with this ethical principle and
may therefore not be a good basis for ethical arguments about how self‐driving
cars should be programmed.
3.49 Incidentally, it is interesting to note here that, in a way, this scenario played
out in real life around 2016. A representative of Mercedes, Christoph von Hugo,
was interviewed during an auto‐show in Paris. When asked about how their self‐
driving cars should be programmed to respond to accidents, Mr. von Hugo
answered that Mercedes' cars would always prioritize saving their owners. Given
people's above‐discussed attitudes suggesting that they would prefer buying such
a car, one might have predicted that this would go over well with people.9
However, there was an outcry. And von Hugo had to later retract his previous
statements. Mercedes ended up claiming that von Hugo's previous statements—
which included some off‐the‐cuff arguments for why it would be a good idea to
always prioritize the owner of the car—were taken out of context. Mercedes,
they now said, had certainly not made up their minds to program their cars to
always prioritize their owners over other people.10
3.50 What should one make of this? Perhaps the people who were outraged
about what Christoph von Hugo had said were thinking of themselves as not
being among those who would buy self‐driving Mercedes cars. This would be
consistent with people's apparently widely shared attitudes of wanting others (in
this case Mercedes‐owners) to have harm‐minimizing cars.
3.51 In the end, this can be taken to be a further indication that what we need are
neither intuitive reactions to cartoonlike vignettes nor off‐the‐cuff arguments
quickly formulated when we are put on the spot (as in von Hugo's case). In
technology ethics, what we instead need are carefully thought‐out arguments that
can be fully articulated and critically assessed by all who participate in the
relevant debates about technology ethics.
3.6 Applying Traditional Ethical Theories
3.52 One way of formulating more fully spelled‐out arguments about how
technologies should behave—for example, about how self‐driving cars should
crash in risky situations—is to draw on the sorts of traditional moral theories that
we looked at in Chapter 2. This way of applying general ethical theories to
specific cases is sometimes called a “top‐down approach.” That is, we can
consider what utilitarians (or consequentialists more broadly), Kantians, virtue
ethicists, or perhaps what those who defend Confucian or Ubuntu‐inspired
theories, would say. Or we could use other ethical theories than the ones we
considered in Chapter 2. For example, we could make use of what is sometimes
called “contractualist” ethical theories. This is something that some philosophers
have done. So let us briefly introduce that ethical theory.
3.53 According to one common form of contractualism, ethics is about
formulating guidelines that people would be willing to adopt as a shared set of
rules, based on non‐moral or self‐interested reasons, in a hypothetical scenario
where they would be making an unforced agreement about how to live together.
In other words, we can imagine that we had to start over again with ethics and
decide on what ethical norms we should have. What would we then agree on, if
we tried to find ethical principles that it would be mutually advantageous for all
of us to have? That's the test that contractualists suggest that we should think in
terms of when we reflect on what actual ethical principles we should accept.
Those principles could then be applied to more specific cases, such as cases
involving crashes with self‐driving cars in accident scenarios.
3.54 We already saw in Chapter 2 what the other ethical theories say. But here is
a quick reminder: utilitarian ethics is about maximizing overall happiness, while
minimizing overall suffering. Kantian ethics is about adopting a set of basic
principles (“maxims”) fit to serve as universal laws, in accordance with which
all are treated as ends‐in‐themselves and never as mere means. Virtue ethics is
about cultivating and then fully realizing a set of basic virtues and excellences.
Confucian ethics is similar to virtue ethics, and also places a lot of emphasis on
the creation and maintenance of social harmony. Lastly, Ubuntu ethics, which we
also mentioned above, is about relating to each other in communal ways that
allow us to fully realize our humanity. What arguments, it might be asked, could
we formulate about crashes with self‐driving cars using these theories? More
generally, is it always possible to apply such traditional theories to issues within
technology ethics in a “top‐down” sort of way?
3.55 Not all of these theories have been applied to the topic of how self‐driving
cars should behave in crash scenarios in particular, at least not so far as the
author of this book is aware. Nor is it obvious that all of these theories can
directly be applied, in a top‐down sort of way, to issues such as that of how self‐
driving cars should behave in crash scenarios.
3.56 For example, while traditional southern African Ubuntu ethics offers an
inspiring and compelling ideal for how human beings should relate to each other,
it is not obvious how one might apply such a theory to the new case of how
machines should behave around human beings whose lives are at risk. As noted
in Chapter 2, Ubuntu ethics has increasingly started making a mark within
technology ethics. And we will get back to it later in the book when we discuss
whether robots could ever have moral rights. But in this context the theory has
not been discussed much. Actually, it is more generally the case that at the time
of writing, few articles have been published that (1) explicitly endorse one of the
traditional moral theories and then (2) apply that theory to the issue of crashing
self‐driving cars in a top‐down way.
3.57 Two significant exceptions are papers by Jan Gogoll and Julian Müller, on
the one hand, and by Derek Leben, on the other hand. Those papers can be
classified as employing forms of contractualist moral reasoning in defense of
specific conclusions regarding accident programming of self‐driving cars. But
most other papers that discuss the above‐mentioned moral theories do so in a
more exploratory way. They investigate what these theories might imply about
self‐driving cars, while often raising worries about using these different theories
in such ways. Or they investigate whether it would be at all possible to program
a self‐driving car using algorithms based on the traditional moral theories.
3.58 For example, in a thorough investigation of the ethics and law of
autonomous driving, the very inventive legal scholar Jeffrey Gurney first
imagines a series of ethical dilemmas of the sorts discussed above. He then
investigates what utilitarians and Kantians, as he understands them, would say
about those dilemmas. The philosophers Patrick Lin and Noah Goodall do
similar things in their early papers about self‐driving cars. We can here focus on
what Gurney says about utilitarian ethics to illustrate that what the moral
theories imply about the choice of accident programming is not always a
straightforward matter. Rather, it is something that is up for debate.
3.59 Gurney suggests that a self‐driving car could be equipped with a powerful
computer enabling it to make utilitarian calculations about the expected utility of
different options in a much quicker and more reliable way than a human could
ever do. Accordingly, Gurney argues that utilitarians would recommend that
self‐driving cars be equipped with these capabilities, and that they would be
programmed to always crash in ways that maximize overall expected utility, for
example, by minimizing overall harm.
3.60 However, it is not altogether clear that this is what a utilitarian would
necessarily recommend. A utilitarian would be mindful of the fact that people
might be scared of taking rides in “utilitarian” cars, instead preferring cars
programmed to prioritize their passengers. After all, this is what was indicated
by the surveys from Bonnefon and his colleagues described above. A clever
utilitarian might take this into account, and recommend that self‐driving cars be
programmed to always save their owners. The utilitarian might recommend this
if it were the case that having a maximum of people willingly using self‐driving
cars rather than regular cars would be likely to bring down the overall number of
deaths and injuries in traffic. Utilitarians would recommend whatever solution
would best promote overall happiness. This might mean that people must be
lured into self‐driving cars by the promise that their cars would be programmed
to behave in “non‐utilitarian” ways in crashes.
3.61 This also highlights that we need to consider who exactly is the moral agent
who needs to make a choice. Is it the car itself? Perhaps its best way of
maximizing utility would be to “tell” people that it is programmed in whatever
way they prefer, but then actually crash in whatever ways would maximize
utility. Or is the moral agent the person designing the car? Or perhaps the
regulatory body permitting certain types of cars on the road? When we try to
apply traditional moral theories to this problem, it is useful to ask ourselves who
exactly the moral agent is who is making the decisions about how self‐driving
cars should crash.
3.62 The same would apply to Kantian ethics: who is it that should choose
maxims they would be willing to lay down as universal laws? Who should treat
everyone as an end and never as a mere means? Is it the technology itself, in this
case the self‐driving car? Or is it the user? Or perhaps the designers of the
technology? Different answers to such questions might lead to different concrete
ethical conclusions about what self‐driving cars or other technologies should do.
3.63 Another issue here is whether we need to choose among these different
moral theories, such that we can either use utilitarian or Kantian or virtue ethical
or contractualist reasoning. Or perhaps we should turn to Confucian or Ubuntu‐
inspired ethical theories. One view would be that we have to make a choice and
that we cannot draw on more than one type of reasoning here. Another view,
which might be more promising, is that there are lessons to learn from all of
these different perspectives.
3.64 For example, the lesson to be learned from utilitarian ethics might be that
when we think about how self‐driving cars should crash, we should do this partly
with an eye to what would be best for the overall good, and everyone's
happiness. The lesson from Kantian ethics might be that we should choose rules
we would be willing to have as universal laws applying equally to all—so as to
make everything fair, and not give some people an unjustified advantage in
crash‐scenarios. The lesson from Confucian ethics might be that our ways of
programming self‐driving cars should promote social harmony. And the lesson
from Ubuntu ethics might be that our ways of programming self‐driving cars
should be consistent with having people relate to each other in ways that enable
them to promote each other's humanity, whatever that might mean in this
context. Perhaps by putting all of these ideas together, we can formulate a
general, higher‐order ideal of how to introduce technologies into human society.
3.65 What about virtue ethics? It is hard to come up with any virtue ethical ideas
about how self‐driving cars should crash, just as it can seem hard to apply
Confucian ethics or Ubuntu ethics to this case in a top‐down way. But virtue
ethics, and those other theories as well, might help when we think about the
ethics of automated driving more generally. A paper by Mark Coeckelbergh
about self‐driving cars and what impacts our experienced sense of moral
responsibility might be relevant here.
3.66 In making his argument, Coeckelbergh uses the sort of “post‐
phenomenological” approach to technology that we briefly looked at in Chapter
1. Using the idea that technologies shape how we experience the world and what
we think we can do, Coeckelbergh argues that how careful people are while
using their cars and how responsible they feel about their car usage both depend
on the design of the cars they are using. Why is this observation relevant to
virtue ethics? Well, being careful and taking responsibility for one's actions are
virtues that we value in people who use risky technologies like cars. So perhaps
a lesson from the virtue ethical perspective is that we should try to design and
program cars and other technologies in ways that help to make people act
carefully and responsibly when they use these technologies.
3.67 Those are all hypothetical suggestions about how one might use traditional
moral theories to formulate arguments in this discussion. But as was mentioned
above, there have also been contributions that take a more firm stance in favor of
certain types of contractualist reasoning about this topic. Here are two examples.
3.68 First, Gogoll and Müller discuss the question of what we all prospectively
have self‐interested reasons to prefer in this domain. This is precisely the type of
question a contractualist would want us to ask about this topic. What Gogoll and
Müller argue using this type of argument is that we all have self‐interested
reasons to want everyone to use cars programmed to minimize harm in crash‐
scenarios. This is a contractualist justification since the justification is not based
directly on the moral value of minimizing harm, but rather indirectly on the self‐
interested value for each person of maximizing their own chances of surviving
any accidents they might be involved in.
3.69 Second, another argument drawing on the contractualist tradition is featured
in a paper by the American philosopher Derek Leben. Leben presents an
argument in favor of a “Rawlsian” accident‐algorithm. John Rawls was a highly
influential political philosopher who argued, among other things, that we can
determine what is just by considering what people would choose if they were
making choices behind a “veil of ignorance” where they did not know what
place they have in society. In such a situation, Rawls thinks that we would all
favor making things as good as possible for those who are the worst‐off
members of society. Leben applies this line of reasoning to self‐driving cars.
Behind a veil of ignorance, Leben argues, it would be rational to choose
accident‐algorithms that would protect whoever is most vulnerable in any
accident scenario. That way, we would make things as good as possible for
whoever is worse off in a crash. This is another type of contractualist
argument.11
3.70 What should we think about these arguments? In a response to Leben's
paper, Geoff Keeling argues, first, that Leben has not given us a correct
interpretation of how John Rawls (or somebody like Rawls) would reason about
this topic. Second, Keeling also argues that Leben's most important decision‐
theoretical claim—namely, that it is prospectively rational to favor a “Rawlsian”
accident‐algorithm—does not correctly identify what it is rational to choose
from a decision‐theoretical perspective.
3.71 Here it is perhaps less interesting to reflect on whether Leben's argument is
a good interpretation of Rawls. It seems more important to consider whether the
argument itself is a good one on its own terms. One possible objection to both
Leben's argument and Gogoll and Müller's argument would be that we should
not accept the basic contractualist premise that moral arguments should be made
by first setting aside moral values and then asking what it would be rational to
choose for self‐interested reasons in some hypothetical contracting scenario. It
may seem better to formulate arguments that explicitly refer to distinctly moral
values or principles. One reason for this is that after a crash, when the outcome
is evaluated, people will be wanting to know if there was any clear moral value
or principle that could be used to directly justify the way the crash happened.
3.72 Much more can be said here. But we will leave it at that. The aim of this
section has not been to take a stance on which of the different traditional ethical
theories does the best job in discussions about how to program or design self‐
driving cars. The aim has instead been to illustrate how one might go about
making use of the traditional ethical theories when thinking about this issue. As
we have seen, there can be room for disagreement about this. The topic of what
traditional theory does the best job—and whether we have to make a choice, or
whether we can combine insights from the different theories—is still an on‐
going discussion within the ethics of self‐driving cars.
3.7 Which Method(s) Should We Use in
Technology Ethics? Only One or Many?
3.73 Above, we have looked at a number of different methods that can be used in
technology ethics. On the most general level, we saw that philosophers tend to
use what is called the “reflective equilibrium” method. That is, they try to
achieve coherence between general ideas and principles, on the one hand, and
more specific ideas and judgments about cases, on the other hand. This can
almost be seen as a form of “meta‐method,” that is, an overarching method that
most researchers who engage in general theorizing are using, where more
specific methods also become necessary. We have looked at a number of such
more specific methods that have been used in the discussion about ethical
accident algorithms for self‐driving cars and related issues.
3.74 Among other things, we have seen that sometimes ethics researchers use
what we called the “ethics by committee” method: gathering a number of experts
who are supposed to formulate and agree on a set of ethics guidelines. We have
also looked at the following methods: comparing technology ethics issues to
classical topics within philosophy (e.g. the trolley problem), empirical ethics that
involves investigating patterns in ordinary people's ethical intuitions, and the
top‐down method of applying traditional moral theories to issues within
technology ethics.
3.75 It is important to note that these are not the only methods there are. Other
approaches have also been suggested and used in this discussion. For example,
the ethics and public policy researcher Johannes Himmelreich has argued that we
can make use of methods developed within political theory when we think about
technology ethics issues, such as the issue of how self‐driving cars should handle
crashes. In a similar spirit, Filippo Santoni de Sio has suggested that we might
take well‐established types of reasoning from legal theory and make use of them
in the context of technology ethics. For example, we might do this by using the
legal “doctrine of necessity” to come up with arguments about how self‐driving
cars should behave in crash scenarios.
3.76 The technology researcher Andreia Martinho and her colleagues, in turn,
have used the interesting approach of systematically comparing the ethical issues
that academic technology ethics experts tend to discuss, on the one hand, and the
ethical issues that receive the most attention from the technology industry,
including the automotive industry, on the other hand. In addition to being
fascinating in itself from a sociological point of view, such an approach could
help us to understand different mindsets and what happens when people with
these different mindsets approach technology ethics issues, such as the issues
related to self‐driving cars.
3.77 These examples above are not the only further methods that can be used in
technology ethics. There are others as well. Given this plurality of possible
methods, the following questions arise. If there are so many different possible
methods one can use when one is doing research in technology ethics, how does
one choose what methods one should use? Are there certain methods that should
perhaps be avoided? More generally, should we be monists about methodology
—that is, should we only favor one particular type of methodology? Or should
we be pluralists about methodology— that is, should we make use of multiple
methods?
3.78 As was noted above, there are some ethics researchers who are strongly
opposed to some of these methods. In particular, when it comes to the topic of
the ethics of self‐driving cars, two methods have been very strongly criticized.
Some have even argued that those methods are ethically questionable. The two
methods in question are the method of comparing the ethics of self‐driving cars
to the trolley problem and the empirical ethics method that is inspired by the
trolley problem.
3.79 In particular, the philosophers Nassim JafariNaimi and John Harris have
both responded to these approaches in very sharp terms. They argue that there is
something overly simplifying or reductive, and also potentially flippant or
insensitive, about comparing real life‐and‐death issues to thought experiments
presented with cartoon‐like vignettes with sometimes comical or absurd
situations being depicted.
3.80 JafariNaimi summarizes her overall criticism of these just‐mentioned
methods in the following way:
First, ethical situations are marked by a deep sense of uncertainty and an
organic character. Second, our place within ethical situations matters greatly.
Third, the impact of our actions in response to ethical situations is not limited
to immediate outcomes … consequences are broad and long ranging.
Therefore … principles that appear to solve the scenarios of experimental
ethics may or may not serve similar ethical situations encountered in real life.
(JafariNaimi 2018 : p. 306)
There is a further problem, JafariNaimi adds, with the whole idea of comparing
the ethics of self‐driving cars with the trolley problem. The problem is that
coming up with inventive and sometimes comical scenarios involving crashing
self‐driving cars and then asking who should live or die is disrespectful of
human life. It is morally insensitive. Or so JafariNaimi thinks.12
3.81 Similarly, Harris argues that life and death decisions are very serious
matters—matters that are not taken seriously enough in the trolley problem and
empirical ethics methodologies that he is criticizing. The examples used in these
methods, Harris further argues, seem to put machines into the role of judges who
“punish” certain people by “condemning them to death.” This is something that
Harris thinks that a machine with artificial intelligence should never be allowed
to do. In summary, then, according to these criticisms, some methods are not
sufficiently serious‐minded, and they threaten to trivialize important real‐life
issues.
3.82 Let us suppose that we take these kinds of criticisms very seriously. One
way of going here would be to try to “cancel” certain methods, so to speak—in
other words, to stop using them ourselves and to try to discourage others from
using those methods. Another way of going would be to instead view these kinds
of criticisms as helping to clarify what matters most in real‐life technology
ethics.
3.83 As noted at the beginning of this chapter, what methods to use or not to use
is itself a philosophical and perhaps also ethical issue in itself. So it is not to be
expected that researchers working on technology ethics are all going to agree
what methods should be used. One potential way out here is to zoom out a little
and to compare, on the one hand, the whole community of researchers working
on technology ethics, and, on the other hand, individual researchers who work
on technology ethics. If we are focusing on the bigger picture, it can seem hard
to deny that there is a value in having different contributors to the research
community use different methods, so that people interested in technology ethics
can learn about what results, findings, or conclusions people come to when they
use different methods.
3.84 We can view technology ethics—or any other field—as a collective
endeavor where there can be a useful division of labor, where some people work
with certain methods or approaches and other people work with other methods
and approaches. Everyone doing research in technology ethics can benefit from
the insights—and mistakes—of other researchers, who use different methods. If
individual researchers wish to avoid certain methods, others can take up the task
of using those methods instead. And the community as a whole can accumulate
and generate different outputs that can be of interest to anybody wishing to
grapple with or learn about technology ethics.
3.85 Individual researchers come in at least two different kinds. There are those
who are only interested in very narrow and specialized topics, and who only
want to use very specific methods when exploring those topics. And there are
those who prefer to work on a wider range of topics, and who might also wish to
explore different methods. Of course, there are also those who do a little bit of
both. Whatever one's approach is, one can certainly always learn from others,
who use different methods or who explore different problems.
3.86 By oneself also occasionally trying out different methods than those that
one is used to or perhaps naturally drawn to, one can also gain new insights and
perhaps new inspiration in one's thinking. Being broad‐minded and open‐minded
are important virtues for people who are thinking about research and academic
topics, and also important virtues more generally in life as well.
3.87 The rest of the book will continue on in that sort of pluralistic spirit: it will
consider different kinds of issues within technology ethics, and present and
assess different kinds of arguments, based on different kinds of methods. Again
and again, we will be encountering disagreements—sometimes very passionate
and spirited ones. As noted earlier, some find this to be a frustrating aspect of
discussions about ethics, while others find it exhilarating.
3.88 The aim of this book is not to argue in favor of any particular conclusions
or any particular methods, but to introduce the reader to the sorts of issues and
the types of arguments that play important roles within technology ethics.
Accordingly, it makes sense to be very pluralistic and to consider very different
points of view—whether it is about methods or any other issues—in a book like
this one.
Annotated Bibliography
Edward Awad, Sohan Dsouza, Richard Kim, Jonathan Shulz, Joseph Henrich,
Azim Shariff, Jean‐Francois Bonnefon, Iyad Rahwan (2018). “The Moral
Machine Experiment,” Nature, vol. 563, pp. 59–64.
This article describes the trolley problem‐inspired “moral machine
experiment,” which studied millions of people's attitudes about how self‐
driving cars should respond to dangerous situations.
Norman Daniels (2020). “Reflective Equilibrium,” in The Stanford Encyclopedia
of Philosophy (Summer 2020 Edition), edited by Edward N. Zalta,
https://plato.stanford.edu/archives/sum2020/entries/reflective‐equilibrium.
An overview of the philosophy of the reflective equilibrium method.
David Edmonds (2013). Would You Kill the Fat Man? Princeton: Princeton
University Press.
An entertaining and informative book about the trolley problem and its
history.
Luciano Floridi (2019). “Translating Principles into Practices of Digital Ethics:
Five Risks of Being Unethical,” Philosophy & Technology vol. 32 no. 2, pp.
185–193.
A brief article describing fives types of risks that arise when attempts are
made to create technology ethics guidelines: ethics shopping, ethics blue‐
washing, ethics lobbying, ethics dumping, and ethics shirking.
John Harris (2020). “The Immoral Machine,” Cambridge Quarterly of
Healthcare Ethics, vol. 29, no. 1, pp. 71–79.
A critical response to the use of the trolley problem in the context of real‐
world ethics of self‐driving cars.
Nassim JafariNaimi (2017). “Our Bodies in the Trolley's Path, or Why Self‐
Driving Cars Must *Not* Be Programmed to Kill,” Science, Technology, &
Human Values, vol. 43, no. 2, pp. 302–323.
Another critical response to the use of the trolley problem in the context of
the real‐world ethics of self‐driving cars.
Andreia Martinho, Nils Herber, Maarten Kroesen, and Caspar Chorus (2021).
“Ethical Issues in Focus by the Autonomous Vehicles Industry,” Transport
Reviews, vol. 41, no. 5, pp. 556–577.
A fascinating comparison between the ethical issues that academic
researchers discuss in relation to self‐driving cars, on one hand, and ethical
issues that are discussed in industry documents in relation to self‐driving cars,
on the other hand.
Sven Nyholm (2018). “The Ethics of Crashes with Self‐Driving Cars: A
Roadmap,” Philosophy Compass, vol. 13, no. 7, pp. 1–20.
Part I: https://onlinelibrary.wiley.com/doi/full/10.1111/phc3.12507. Part II:
https://onlinelibrary.wiley.com/doi/full/10.1111/phc3.12506. This two‐part
article provides a roadmap of the key issues within the ethics of self‐driving
cars. It also contains the references to all of the research articles discussed in
this chapter that are not listed here in this short annotated bibliography.
Notes
1 For an example of how philosophers thought about self‐driving cars around
that time, check out his video created by Patrick Lin: “The Ethical Dilemma
of Self‐Driving Cars”:
https://www.ted.com/talks/patrick_lin_the_ethical_dilemma_
of_self_driving_cars
2 For more information, see the Wikipedia article on “Death of Elaine
Herzberg”: https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg
3 EU's high‐level expert group's “Ethics Guidelines for Trustworthy AI” can be
downloaded here: https://ec.europa.eu/futurium/en/ai‐alliance‐consultation.
1.html. The German “Ethics Commission's Complete Report on Automated
and Connected Driving” can be found in an English translation here:
https://www.bmvi.de/SharedDocs/EN/publications/report‐ethics‐
commission.html?nn=187598
4 Metzinger presents his criticisms in his 2019 article “Ethics Washing Made in
Europe,” in Der Tagesspiegel, available here:
https://www.tagesspiegel.de/politik/eu‐guidelines‐ethics‐washing‐made‐in‐
europe/24195496.html
5 Here is an episode of the “Philosophy Bites” podcast in which David Edmonds
discusses the trolley problem: “David Edmonds on the Trolley Problem”:
https://philosophybites.com/2013/09/david‐edmonds‐on‐trolley‐
problem.html. Those interested in a deep‐dive can listen to Frances Kamm's
2013 Tanner Lectures about the trolley problem here: “The Trolley Problem:
May We Harm Some to Save Others?”:
https://tannerlectures.berkeley.edu/2012‐2013
6 You can hear Greene discuss his research on the “Mindscapes” podcast here:
“176: Joshua Greene on Morality, Psychology and Trolley Problems”:
https://www.preposterousuniverse.com/podcast/2021/12/06/176‐joshua‐
greene‐on‐morality‐ psychology‐and‐trolley‐problems
7 https://www.moralmachine.net
8 You can see Bonnefon discuss the work of his research team here: “Jean
François Bonnefon—The Moral Machine Experiment”:
https://www.youtube.com/watch?v= Nq2i6VdITJA. Here, also, is a video of
Bonnefon's colleague Iyad Rahwan giving a presentation on this topic: “What
Moral Decisions Should Driverless Cars Make?|Iyad Rahwan”:
https://www.youtube.com/watch?v=tb‐WdVA4_bo
9 Michael Taylor (2016) “Self‐Driving Mercedes‐Benzes Will Prioritize
Occupant Safety Over Pedestrians,” Car and Driver:
https://www.caranddriver.com/news/a15344706/self‐driving‐mercedes‐will‐
prioritize‐occupant‐safety‐over‐pedestrians
10 Raphael Orlove (2016) “Now Mercedes Says Its Driverless Cars Won't Run
Over Pedestrians, That Would Be Illegal,” Jalopnik:
https://jalopnik.com/now‐mercedes‐ says‐its‐driverless‐cars‐wont‐run‐over‐
ped‐1787890432
11 You can hear Leben discuss his view on the “Machine Ethics” podcast here:
“23. How to Design a Moral Algorithm with Derek Leben”:
https://www.machine‐ethics.net/podcast/23‐derek‐leben
12 You can hear JafariNaimi—who has changed her last name to Parvin since
publishing her 2018 article—discuss her work here: “Cars, Fashion, and the
False Promise of Algorithmic Judgment|Nassim Parvin|Design@Large”:
https://www.youtube.com/watch?v=lc_bUg0apvI
4
ARTIFICIAL INTELLIGENCE, VALUE
ALIGNMENT, AND THE CONTROL PROBLEM
4.1 Averting a Nuclear War
4.1 On September 26, 1983, a nuclear attack early‐warning system in the Soviet
Union mistakenly indicated that a nuclear missile had been launched from the
United States. The warning system indicated that the missile was heading in the
direction of the Soviet Union and that the first missile had been followed by five
more. According to Soviet military protocol, it was the job of the duty officer of
the command center to report any such indications of the satellite warning
system to officers higher up on the chain of command. They could then quickly
prepare for war. The responsible lieutenant colonel of the air defense forces—
Stanislav Petrov—decided to disobey his orders, however. He did so because he
deemed this to be a false alarm. If he had relayed the message generated by the
missile detection system in accordance with protocol, this might have triggered a
large‐scale retaliatory nuclear attack by the Soviet Union on the United States. A
nuclear war might have broken out. Because he refused to convey the warning
created by the faulty detection system, Stanislav Petrov is thought to have helped
to prevent a nuclear war, with all of the devastation that it might have caused to
large parts of the world.1
4.2 That was a real‐world example of how a technology almost caused an
enormous problem: a nuclear war. Academic discussions about risks related to
advanced technologies also contain lots of imaginary examples of how
technologies could potentially cause problems. Many such examples in recent
times have been about artificial intelligence (AI) and, in particular, powerful
forms of AI that we do not yet have. For example, in his well‐known 2014 book
Superintelligence: Paths, Dangers, and Strategies, the Swedish‐born Oxford
philosopher Nick Bostrom, who is the head of the “Future of Humanity
Institute,” imagines a much‐discussed and somewhat bizarre scenario.
4.3 Bostrom imagines a super‐intelligent AI system that has a seemingly
innocent goal: to produce paperclips and to create as many of them as possible.
The problem here is that this super‐intelligent system is super‐efficient at
maximizing the number of paper clips. And the AI system has no other ultimate
goals programmed into it, meaning that it will take whatever means necessary to
super‐efficiently fill the world with as many paperclips as possible. The system
quickly realizes that there would be more room for paperclips if human beings
did not take up so much space that could otherwise be used as storage space for
more paperclips. Accordingly, the paperclip‐maximizing super‐intelligent system
will promptly seek efficient ways of removing human beings (and animals and
everything else not needed for producing paperclips) in its pursuit of the goal of
maximized paperclip production. Thus, the innocent goal on the part of a super‐
intelligent machine of creating paperclips could lead to the end of human life on
Earth. And if the AI system were powerful enough, we could quickly lose
control over such a system.2
4.4 Suppose next that the goal is not to maximize paperclip production, but
rather to promote human health and longevity. If a super‐intelligent and super‐
powerful AI system had such a goal, presumably there would be no need to
worry that this technology might lead to the end of humanity on Earth. However,
here too we might have a control problem. It might be that the super‐intelligent
and powerful AI system notices that people are not always acting in their own
best interest. So perhaps the system will need to take control over our lives and
force us to exercise more, eat more healthy food, or engage in fewer dangerous
activities (such as mountain climbing, simply leaving our house, or whatever).
Before we know it, all aspects of our lives could be controlled by this AI system.
And we might have no way, or at least no easy way, of taking back control over
our own lives.
4.5 These last two examples are imaginary thought experiments. The first
example—where a missile detection system almost triggered a nuclear war—
was a real‐world case. All three examples help to illustrate the topics that will be
discussed in this chapter. The chapter will look at the closely related so‐called
value alignment and control problems. These are usually discussed primarily in
relation to AI, and sometimes in relation to an imagined, future superpowerful
form of superintelligence, like in the Bostrom examples above. However,
questions about the alignment of technologies with human values, and worries
about whether we might lose control over some of the technologies we use, are
not limited to the issue of AI.
4.6 Researchers sometimes talk about, or worry about, people getting “addicted”
to certain technologies—e.g., people getting addicted to social media. That is
another potential loss of human control related to technologies. And
technologies that are not advanced forms of AI can also align in better or worse
ways with human values and interests. For example, there was a viral video a
few years ago showing a soap dispenser that automatically reacted by dispensing
soap when a person put a hand underneath it—the only problem being that it
only worked for people with a light skin tone.3 A soap dispenser with a sensor is
not an advanced form of AI. But it is another example of a technology that may
or may not be well aligned with human values and interests.
4.7 This chapter will start with a reminder of what is meant by AI and a general
explanation of the idea of value alignment of AI in particular. The chapter will
then introduce two simple distinctions that are highly useful in ethical
theorizing: the distinction between positive and negative value, on the one hand,
and the distinction between instrumental and non‐instrumental forms of
goodness or badness, on the other hand. This will help us to think about various
different ways in which technologies can be well or poorly aligned with human
values.
4.8 The discussion will then reintroduce the issue of human control over
technologies, and a general explanation of what is called the control problem
will be provided. The chapter will end with discussion of some dilemmas
regarding our human desire to be in control over the technologies we use, which
will relate back to the idea of more or less sophisticated forms of AI, and the
instrumental and non‐instrumental theories of technology from Chapter 1.
4.2 What Is Artificial Intelligence and What Is the
Value Alignment Problem?
4.9 We already briefly looked at the question of what AI is at the end of Chapter
1. However, when it comes to discussing the idea of value alignment, and then
later the control problem, it is good to have a quick reminder of what is meant by
AI. The term itself, as was also noted in Chapter 1, comes from science unlike
the word “robot,” which comes from fiction. However, many ethical discussions
about AI—like the one from Bostrom just mentioned above—come across like
something from out of science fiction rather than something from reality.
Moreover, depending on what time period we are looking at, even the science of
AI is itself sometimes like something out of science fiction, it can also be noted.
4.10 When the term “artificial intelligence” was introduced, for example, it was
as part of a research proposal for a two‐month workshop that was going to take
place at Dartmouth College in 1956. In that proposal, the following very
ambitious goals were set out:
We propose that a 2‐month, 10‐man study of artificial intelligence be carried
out during the summer of 1956 at Dartmouth College in Hanover, New
Hampshire. The study is to proceed on the basis of the conjecture that every
aspect of learning or any other feature of intelligence can in principle be so
precisely described that a machine can be made to simulate it. An attempt
will be made to find how to make machines use language, form abstractions
and concepts, solve kinds of problems now reserved for humans, and improve
themselves. We think that a significant advance can be made in one or more
of these problems if a carefully selected group of scientists work on it
together for a summer.4
Perhaps needless to say, solving the problems set out in this proposal has taken a
little longer than these scientists expected it to. And many of those problems are
still not solved. Some might even be unsolvable. What these scientists very
clearly succeeded in doing, however, was to introduce the term “artificial
intelligence” into our common vocabulary. And they also managed to inspire
both researchers and the general public to become fascinated with this idea.
4.11 In the proposal, AI is understood in terms of the notion that “learning or any
other feature of intelligence” can be simulated by a machine, which would then
possess AI. In other words, we start with whatever we associate with human
intelligence. If we can create a technology that can “simulate” that, then we have
created AI. Others—like the genius mathematician and computer science pioneer
Alan Turing—have talked about machines that are “imitating” intelligent human
behavior and thinking.5 Yet others have talked about machines that are
“replicating” human intelligence.
4.12 So, in order to understand what AI is, we need to reflect on what we
understand by human intelligence in the first place. Here it is useful to
distinguish between what one might call theoretical and practical human
intelligence. Theoretical intelligence is concerned with taking in information,
learning, understanding, the capacity for analysis, and other theoretical ways of
being intelligent. Practical intelligence—including emotional intelligence—is
concerned with action, decision‐making, planning, cooperation with other
agents, and other more practical ways of displaying intelligence.
4.13 So if machines can simulate human or greater levels of intelligence, they
might have one or the other, or both of, theoretical and practical intelligence. A
self‐driving car, for example, requires practical intelligence: it needs to be able to
act, make decisions, and respond intelligently to its environment in the pursuit of
the goal of getting around in traffic. A computer program tasked with helping to
diagnose cancers, in turn, primarily needs to have a more theoretical form of
intelligence: it needs to be able to take in information, identify key properties,
and then analyze that information.
4.14 Speaking of goals, it is noteworthy that intelligence—and therefore also AI
—is sometimes simply defined as the capacity to efficiently achieve goals in a
range of circumstances. But this seems to overemphasize the practical
dimensions of intelligence. It does not seem to sufficiently take into account a
more theoretical form of intelligence. There is no clear reason why we should
emphasize one form of intelligence more than any other, whether we are talking
about human beings or technologies. Ultimately, it seems best to view
intelligence as a complex phenomenon, with different parts to it, some of which
are more theoretical, some more practical.
4.15 Let us, however, stay with the idea of practical intelligence and the capacity
to efficiently achieve goals for the moment. This form of intelligence is helpful
to refer to when one explains the difference between so‐called weak AI, on the
one hand, and strong AI, on the other hand. Another pair of terms that is
sometimes used to express this distinction is “narrow” and “general” AI.
4.16 Most technologies with any form of intelligence have a weak or narrow
form of AI. This means that they are only able to carry out certain specific tasks,
and that they are only able to do so in certain specific conditions. This makes
them different from human beings, who are able to pursue a wide range of goals,
in a flexible and creative way, across a wide range of different circumstances. A
technology that had such a capacity would have strong or general AI. This is
sometimes referred to as artificial general intelligence or AGI. Notably, experts
disagree about when it might become possible, and whether it is at all possible,
to create AGI. What Bostrom refers to by the label “superintelligence,” in turn,
would be a form of AI or AGI that is extremely efficient in accomplishing
whatever goals it might have, in a very flexible and highly creative way.
4.17 The idea of “value alignment” of AI tends to come up in discussions of AI
technologies that are capable of pursuing goals in more or less autonomous
ways. The idea is that we should make sure that technologies equipped with AI
function in a way that is aligned with, or fits with, human values. Strictly
speaking, it is not only AI that should ideally be aligned with human values. Any
technology, whether or not it has AI, should ideally promote, respect, or
otherwise harmonize with human values. This is sometimes discussed under the
heading of “value sensitive design.” This is the idea that the design of
technologies should always be sensitive to the values that are at stake in the
domains where the technologies in question are used. But let us stick with AI for
the moment, and let us say something more about the value alignment challenge
as it applies to AI.
4.18 One useful source on how to understand the idea of value alignment is a
paper written by Iason Gabriel, who is a political theorist and ethicist working
for the AI company “DeepMind” in London. In his paper, Gabriel uses a goal‐
focused understanding of intelligence according to which it refers to “an agent's
ability to achieve goals in a wide range of environments” (Gabriel 2020: p. 412).
With this understanding of intelligence in mind, Gabriel explains that the
challenge of achieving value alignment of AI has two distinct parts.
4.19 The first part is the technical challenge of encoding or embedding values or
other principles of ethics into technologies in such a way that they reliably do
whatever they ought to do. One challenge here is to get technologies to do all
that they are supposed to do (e.g., without falling over, exploding, or whatever).
Another challenge is to avoid so‐called “reward hacking,” which means that the
AI technology achieves its goal, but that it does so in some way that involves
taking some form of unexpected shortcut, or that is otherwise not what we might
have had in mind.
4.20 The second key part of the alignment problem is what Gabriel calls the
normative problem of specifying what the right values or ethical principles are
that AI systems (or other technologies) should align with. For example— to refer
back to some of the theories discussed in Chapters 2 and 3—should these
technologies align with utilitarian, Kantian, virtue ethical, or perhaps traditional
Confucian or Ubuntu values? Or some other set of values, or some combination
of these values and principles, perhaps? Related to that question is the question
of whether AI should align itself with some particular value in a minimal or
maximal way. The former might mean that the technology is safe but does not
perform optimally with respect to whatever value is in question. The latter might
mean that the technology performs optimally with respect to some specific
value, but that it creates trouble for other human values and goals, for example
by getting out of control.
4.21 In his intriguing discussion, Gabriel goes on to explore not only the ethical
question about what values and principles AI should align with, but also the
political question of who should decide what these values should be. He also
considers how we could go about finding some sort of compromise among
different perspectives, which treats everyone equally and fairly. That is an
interesting further discussion—and certainly one that is very important as well.6
4.22 But we will here stick with the ethical dimensions of the analysis, and
consider how some very basic distinctions typically drawn in ethical theory can
be used to introduce some further interesting angles on the idea of value
alignment. We will do so in a way that will end up tying back this discussion of
value alignment to the different conceptions of technology that we looked at
earlier in Chapter 1. We will also zoom out a little bit and not only talk about the
alignment of AI with values, but the alignment—or potential misalignment— of
technologies more generally with human values.
4.3 The Good and the Bad, and Instrumental and
Non‐Instrumental Values and Principles
4.23 One very basic distinction that is central to a lot of ethical analysis is the
distinction between instrumental and non‐instrumental or intrinsic values. While
much day‐to‐day decision‐making concerns itself with instrumental values—that
is, the question of which the best means to certain ends are—most philosophical
theorizing about values tends to be about non‐instrumental or intrinsic values.
The latter are things that are not means to ends, but that are ends or goals in
themselves. Things, in other words, that are regarded as valuable or important
for their own sake.
4.24 Money, for example, is usually thought of something that is mainly
instrumentally valuable: you can use it as a means of acquiring other things,
which you might value for their own sake. Indeed, money is sometimes said to
have its value by being a form of universal means, which can get one all sorts of
things that it might be good to have. Some other things, in contrast, are typically
seen as goals or ends in themselves. Examples include friendship, love, mutual
respect, human achievement, knowledge, wisdom, justice, beauty, moral virtue,
well‐being, and life itself. These are things that tend to be valued, chosen,
desired, celebrated, and so on for their own sake.
4.25 People and animals are also usually seen as valuable or important in
themselves. Recall, for example, Kant's ethical principle that says that we should
always treat the humanity in each person as an end in itself, and never merely as
a means. Kant himself did not view animals in the same way. But many people
would also say that some, or perhaps all, animals have a value of a non‐
instrumental form. The contemporary Kantian philosopher Christine Korsgaard,
for example, thinks that we should also treat animals as ends in themselves, and
never merely as means.7 The same is sometimes said about the natural
environment. It is also sometimes seen as valuable in its own right. These are all
examples of things that are commonly seen as having a non‐instrumental or
intrinsic value. Things that have instrumental value—such as tools, medicines,
money, some technologies, exercise, nutrition, and so on—are usually seen as
having their value as means because they help to protect, promote, or otherwise
positively relate to what is seen as having non‐instrumental value.
4.26 In the two paragraphs above, there is only talk of things that have positive
value. But we also concern ourselves, both in ethical theory and in life in
general, with things that have a negative value. That is, things that are bad,
horrible, ugly, regrettable, shameful, unethical, or otherwise negative in some
way or other. Here, too, we can distinguish between things that are
instrumentally bad and things that are non‐instrumentally bad. The former are
things that have bad effects or that lead to bad outcomes. The latter are things
that are seen as bad or negative in themselves. Examples of things that are often
seen as bad in themselves in a non‐instrumental way include suffering, hate,
stupidity, disrespect, vice, slavery, injustice, prejudices, meaninglessness, and—
according to some—death.
4.27 With these basic and yet crucial distinctions, we can create the following
classification matrix:
Value Positive
Negative
s and
other
ethica
l
consi
derati
ons
Instru Instrumental positive values: what is good Instrumental negative value:
menta as a means to other ends
what has bad effects or leads
l
to undesirable outcomes
Non‐ Non‐instrumental positive values: things Non‐instrumental negative
instru that are good or valuable in themselves, values: things that are bad or
menta as ends or goals, such as persons, animals, regrettable in themselves,
l
the natural environment, friendship, love, such as suffering, hatred,
mutual respect, justice, beauty,
achievement, virtue, well‐being, and so
on
vice, stupidity, prejudices,
injustice, death, and
destruction
4.28 Notably, when the topic of value alignment of AI is discussed, this is
typically and primarily done from an instrumental point of view. The question is
usually whether AI might promote its assigned goals in a way that leads to
positive or negative outcomes. Perhaps this focus is usually the one that is taken
because many researchers who are interested in AI value alignment have a
primarily instrumental understanding of technology, more generally. However, it
can also be interesting to think in terms of non‐instrumental value in this context.
4.29 In other words, can the operation of AI—or the functioning of any other
technology—sometimes be non‐instrumentally good or bad? This is a question
that not only arises from the consideration that there are both instrumental and
non‐instrumental values. It also arises since there are instrumental and non‐
instrumental ways of conceiving of some technologies, as we saw in Chapter 1.
So, let us reflect on the idea of value alignment of AI and technology more
generally, not only with instrumental positive or negative value in mind, but also
with an eye to non‐instrumental positive or negative value—especially as this
might relate to non‐instrumental ways of understanding some technologies.
4.30 Accordingly, at least in theory, we have the following possibilities:
Value‐
Positive
Negative
alignment or
misalignmen
t of
technologies
Instrumental The technology reliably and The technology has, or potentially
robustly has good effects
has, bad effects
Non‐
The technology—or the way The technology—or the way the
instrumental the technology operates—is technology operates—is somehow
somehow good in itself, as bad in itself, or represents
an end or as a goal
something that is bad in itself
Let us now explore these possibilities, to see if they also have some potential
application in practice and not only in theory. As we do so, some of the
terminology used in the boxes above will be explained.
4.4 Instrumentally Positive Value‐Alignment of
Technologies
4.31 In the box related to instrumentally positive value alignment in the matrix
above, it says that technologies that fit into that box should “reliably” and
“robustly” have good effects. Those are the most interesting forms of
instrumentally and positively value‐aligned technologies. Why? The thought is
that it should ideally not be just a one‐off fluke or happy accident that a
technology has good effects. It should be something reliable in the sense that we
can again and again rely on the technology's delivering the goods over time, so
to speak. And it should ideally be a robust delivery of the goods, in the sense
that the technology produces good effects across a wide range of circumstances,
not just in certain ideal conditions that may be hard to replicate in real life.
4.32 Let us again use the example of the self‐driving car, as it is often discussed
by those who have high hopes for that technology. There is, for example, a very
optimistic Ted Talk from 2015, by Chris Urmson, who was then the head of the
technology company Google's self‐driving cars research program. In that video
—“How a Driverless Car Sees the Road”8—Urmson is about as enthusiastic
about the positive potential of self‐driving cars as anyone could possibly be. On
the one hand, Urmson offers statistics about how dangerous, but also time‐
wasting, it can be to drive regular cars. On the other hand, he describes a
projected future in which self‐driving cars will very seldomly, if ever, be
involved in serious car accidents, and where we can do all kinds of productive
things while traveling from A to B.
4.33 Urmson also describes how he thinks that self‐driving cars will be able to
deal with the most unexpected and unusual circumstances, not only the most
ideal conditions. For example, Urmson describes an unusual case that his team's
experimental self‐driving car had to react to. A woman in an electric wheelchair
was going around in circles following a mother duck and her group of ducklings,
in the middle of the road. Urmson also describes cases in which people on
bicycles and in manually driven cars act in seemingly unpredictable and
irregular ways. According to Urmson, the experimental self‐driving cars his
team of engineers was developing were able to react in safe and efficient ways
even in these unusual circumstances.
4.34 Let us set productivity and time‐wasting aside and just focus on safety. If
what Urmson describes in that 2015 Ted Talk represents what self‐driving cars
are, or will become, able to do in terms of their safety on the road, this would be
an example of a technology that is reliably and robustly value aligned in the
instrumentally positive sense. This technology would allow us to reliably get to
where we want to go in a safe way, in a wide range of different circumstances,
and not just in the most ideal conditions.
4.35 Of course, it should be mentioned here that many researchers—including
Chris Urmson himself more recently!9—have expressed some skepticism about
how soon, or whether at all, it will be possible to create fully autonomous self‐
driving cars that will very reliably and robustly be value aligned in this positive
instrumental sense. But if this ideal is achieved in the way that Urmson describes
it in the above‐referred to Ted Talk, it would be a very good example of a
technology that is positively value aligned in the instrumental way.
4.36 From a practical point of view, this form of reliable and robust
instrumentally positive value alignment is the form of positive value alignment
that most naturally comes to mind for most people when they think about
technology in terms of positive value. It certainly fits with one kind of
engineering mindset that was described in chapter one: the idea that technologies
can function as powerful and innovative tools that can help to solve concrete
practical problems in efficient ways.
4.37 This fits with the so‐called instrumental theory of technology. But what
about some of the non‐instrumental ways of thinking about technology that were
also described in Chapter 1? We will get to this idea, both from the positive and
negative point of view, in just a moment. But let us first briefly consider the idea
of negatively misaligned technologies in the instrumental sense.
4.5 Instrumentally Negative Misalignment of
Technologies
4.38 As noted above, reflection on positive value‐alignment (in the instrumental
sense) is usually about whether some technology will, with a great deal of
reliability and robustness, operate in a way that has good effects. This allows that
technologies would sometimes fail to produce good outcomes, so long as they
mostly produce good outcomes. In contrast, when it comes to worries about
potential instrumentally negative misalignment of technologies, reflection is
usually not focused—or not only focused—on how technologies work in most
cases. Rather, much reflection is likely to be about risks of bad things that could
sometimes happen if the technologies in question are widely used.
4.39 When people talk about threats to privacy posed by data collecting
technologies, for example, they often do not claim that in every single case that a
technology collects data, some clearly and easily identifiable bad outcome will
be brought about. They usually rather worry that if technology companies have a
lot of personal data about private individuals, this might lead to bad outcomes in
certain circumstances. According to the philosopher Carissa Véliz, this gives
data‐collecting technology companies a form of power over the fate over those
whose personal data they are collecting, which exposes those people to
unacceptable risks, even if the data does not end up being used for any bad
purposes, and no bad effects actually come about in many cases.10
4.40 Similarly, the Soviet Union's nuclear missile detection system that was used
back in the 1980s described in the introduction may seem to have been
instrumentally and negatively misaligned with important values because of how
it could have caused a nuclear war. Had it not been for the fact that Stanislav
Petrov decided to disobey his orders because he thought the nuclear detection
system had made a false alarm, harm on a massive scale might have come about
because of the workings of this technology and the protocol established around
it. This is another case where the likelihood of the undesirable bad outcome
might have been small, but where the bad outcome can be considered bad
enough that it might be seen as making the technology misaligned with the
important value of protecting people's safety from the possibility of a nuclear
war.
4.41 Other technologies might seem to be misaligned with key human values
because they systematically cause problems for certain groups of people. The
example of a soap dispenser with an infrared sensor that only reacted to people
with a light skin tone was one such example. Another example that technology
ethics researchers sometimes discuss is the design of most cars, which are
typically safer for men to drive than for many women, because of what the
distance to the steering wheel and the pedals from the driver's seat is like.
Women tend on average to be shorter than men. And so, if cars are equipped
with safety features modeled on male drivers, this might systematically
disadvantage female drivers, who might then enjoy less protection of their safety
in crashes.11
4.42 Returning briefly to AI, the sorts of examples that are often discussed in the
value alignment literature tend to relate to what Stuart Russell calls the “King
Midas problem.” This is the problem that what might seem like a good and safe
thing to wish for might not in fact be so good if one manages to create an
intelligent machine that tries to achieve the thing in question. King Midas, in the
myth, was granted his wish that everything he touched would turn to gold. This
might seem like a good idea. However, when the food we try to eat turns to gold,
for example, we would quickly realize, as we were unable to eat anything, that it
was not such a good idea to have everything we touch turn into gold. In the same
way, an AI system might very efficiently and reliably—and robustly—manage to
achieve some positive‐seeming goal. But in doing so, it might take means to
these ends that are damaging for human beings and human values.12
4.43 Recall Bostrom's example of the paperclip‐producing AI system. Its being
extremely efficient in maximizing its paperclip output might involve its
removing human beings who take up space where there could be more
paperclips. This would be an example where whoever created this efficient
paperclip maximizer was not sufficiently careful about what they wished for, just
like King Midas who was not sufficiently careful about choosing his wishes.
4.44 It is also important to note that AI does not have to be superintelligent or
superpowerful to potentially cause significant problems. AI‐critical researchers
like Aimee van Wynsberghe and others argue that the technologies involved in
AI systems—at least the ones we have today—are highly unsustainable from an
environmental point of view. They use up massive amounts of resources, and
have an enormous carbon footprint, because of all the computing power needed.
Van Wynsberghe writes:
… the process of training a single, deep learning, natural language processing
(NLP) model (GPU) can lead to approx. 600,000 lb. of carbon dioxide
emissions […]. Compare this to familiar consumption and you are looking at
roughly the same amount of carbon dioxide emissions produced by five cars
over the cars' lifetime. Other studies have shown that “Google's AlphaGo
Zero generated 96 tonnes of CO2 over 40 days of research training which
amounts to 1000 h of air travel or a carbon footprint of 23 American homes”
[…].
(Van Wynsberghe 2021: pp. 213–214)
What should one think of such shocking statistics? Van Wynsberghe comments:
“In a time when the world must commit itself to reducing carbon emissions, one
has to ask if the emissions from algorithms that can play games (or do other
menial tasks) is really worth the cost” (Van Wynsberghe 2021: pp. 213–214).
This can be seen as an instrumentally negative misalignment of AI systems with
key human values.
4.45 We will discuss alignment and misalignment in the instrumental senses
more when we get to the control problem below. For now, let us turn to the issue
of whether—and if so how—there might be value alignment or misalignment of
any interesting non‐instrumental sorts. In other words, the question is: could
technologies, or the ways in which they operate, be in themselves good (i.e., not
only as a means but as an end), or in themselves bad or ethically problematic?
From the point of view of practical, real‐world ethics, this might seem like a
slightly strange question, at least at first. But from a philosophical point of view,
i.e., when we think about what is possible and how things could be, this is a
fascinating question to reflect on. Let us start with the positive case.
4.6 Positive Non‐Instrumental Value Alignment
of Technologies
4.46 Recall that there are philosophers and others who claim that some
technologies can be our friends, or perhaps even our romantic partners. John
Danaher, for example, has argued that robots can—if the AI in these robots is
advanced enough—be our friends. Helen Ryland, somewhat similarly, argues
that while robots cannot be our friends in the fullest possible sense, we should
distinguish among different degrees of friendship, and robots can be our friends
at least to a limited degree. A theologian and philosophy researcher, who
researches “robot theology,” pastor Joshua K. Smith, has even stated that robots
might become “the best friends we could have.”13 David Levy—a computer
scientist‐turned‐philosopher—predicts that within 50 years, human beings will
fall in love with, and want to marry, robots.
4.47 In fact, there are already some people who want to marry technologies and
other inanimate objects. A man living in Michigan, and who goes under the
moniker “Davecat,” has made many media appearances in which he talks about
how he has been married to a doll named Sidore for over 20 years.14 A man in
Japan married a hologram, and a man in China married a robot he built. A
woman in Paris wanted to marry the Eifel Tower, and a woman in Berlin wanted
to marry the Berlin wall. This might sound crazy to many people. But the
philosopher Janina Loh takes a different view. Loh argues in favor of an
“inclusive” approach. Loh thinks that we should not view the tendency to
become attached to technologies as a “shortcoming” but rather as a “capability.”
According to Loh, we should celebrate human diversity in all of its different
expressions. Therefore, we should welcome and celebrate the notion that people
might love or want to be friends with robots and other technologies.
4.48 If these ideas make sense—a topic we will return to in Chapter 9—then we
have cases where technologies seem to acquire a non‐instrumental value in
themselves as ends, at least for some people.15 After all, friendship and the
people who are our friends, and love and our romantic partners, are examples of
what are usually valued as ends in themselves and not as mere means. So, if
robots, chatbots, or other technologies can be our friends, or perhaps even our
romantic partners, then they might be seen as aligning with human values in a
non‐instrumental way, since friendship and friends are non‐instrumentally
valuable from the point of view of our common ways of valuing things.
4.49 Consider next the idea that we might be able to create artificially intelligent
machines that are able to make moral decisions and be “moral agents,” on the
one hand, or that might deserve to be treated with some moral consideration and
become “moral patients,” on the other hand. As noted in the previous chapter,
some argue that self‐driving cars, for example, will need to be able to make
moral decisions in cases where risky traffic scenarios arise that involve moral
dilemmas. Similar claims have been made about autonomous weapons systems
or military robots, which are also thought to sometimes need to make moral
decisions. If we think that being a moral agent and making moral decisions are
non‐instrumentally valuable things, then the moral agency of certain artificially
intelligent machines might also have some form of non‐instrumental value.
4.50 Suppose, for example, that a machine (e.g., a self‐driving car or a military
robot) is faced with some form of moral dilemma (e.g., whether to kill one
person in order to save five people). And suppose that the machine responds to
this moral dilemma in a way that seems to be the right and proper way to
respond to the situation. The machine might then be seen as having “done the
right thing,” perhaps also “for the right reasons,” and the machine might also be
seen as having displayed a form of moral virtue. If that is so, this might be seen
as a way in which the machine has operated in a way that aligns with important
values, not merely in an instrumental way, but in a way that is non‐
instrumentally good. Why? Because doing the right thing, for the right reason,
and being virtuous are among the things that are usually thought to have a non‐
instrumental value in and of itself, and not merely as a means to other ends.
Accordingly, if a machine can be a moral agent that acts rightly and virtuously, it
could be seen as doing something that is good in and of itself.16
4.51 Consider next the idea that machines—e.g., robots with advanced AI—
might come to deserve to be treated with some degree of moral consideration.
The political scientist and legal theory expert Joshua Gellers, for example, thinks
that it might make sense to give rights to robots, just like some people think that
it makes sense to extend rights not only to animals, but also to things like rivers
or ecosystems. John Danaher, who argues that we can be friends with robots,
also argues that if robots behave like humans and animals with moral status
behave, then we should regard those robots as having the same or a similar
moral status as those humans or animals. Philosophers like Eric Schwitzgebel
and Mara Garza, as well as John‐Stewart Gordon, argue that if machines develop
humanlike intelligence, we should view them as having the sort of moral status
that human beings have.
4.52 Even Joanna Bryson, who thinks that “robots should be slaves,” thinks that
if machines were to become intelligent or sentient in ways similar to human
beings, then we would have moral obligations toward these machines. (Bryson,
as we saw in Chapter 1, thinks that we should avoid creating any such machines
toward which we would have obligations, so that machines can always be treated
as mere means to our ends.) The German philosopher Thomas Metzinger, in
turn, thinks that it would be possible to create machines that can suffer, but that
we should not do so since it is immoral to cause suffering.17
4.53 What all of these ideas have in common is that they seem to treat machines
or other technologies as entities with an important moral status, whereby these
technologies would have a value or moral status whereby they would have to be
treated as ends, and not as mere means. Additionally, since moral patients are
regarded as valuable in themselves, and not merely as means, any technology
considered to be a moral patient would be another example of technologies that
would be having a value that is not purely instrumental but that is also of non‐
instrumental importance.
4.54 Moreover, if the ways that some technologies operate would be in
themselves beautiful, if technologies could accomplish great achievements, or if
they would operate in other ways that are considered non‐instrumentally good
when human beings behave in those ways—then those might be further
examples of technologies that align with important values in non‐instrumentally
positive ways.
4.55 It is important to note here that all of the ideas mentioned above—that
technologies can be our friends or romantic partners, that they can be moral
agents or moral patients, and so on—are controversial ideas that are disputed by
many people who do research on these topics. We will later consider these ideas
and investigate both their merits and their weaknesses. For now, the point is just
that if technologies (or the ways in which they operate) can realize non‐
instrumental values in any of these or any other ways, then there can be what we
might call non‐instrumentally positive value alignment, and not only
instrumentally positive value alignment of AI and other technologies.
4.7 Negative Non‐Instrumental Value
Misalignment of Technologies
4.56 Let us return to the example of a soap‐dispenser that only worked for
people with a light skin‐tone. This can be seen as an instrumentally negative
misalignment of a technology with the value of treating everyone equally, as
well as with the value of giving everyone opportunities to wash their hands. But
this might also potentially be seen as an inherently racist technology. At least
that is the view of the philosophers Shen‐Yi Liao and Bryce Huebner, who claim
that there can be such a thing as what they call “oppressive things.” By this they
mean that some technologies or technologies systems might be part of
oppressive social systems, which discriminate against or in other ways harm
specific groups of people.
4.57 Some medical technologies, for example, also work better for people with a
light skin tone because they were also developed with members of groups with
light skin tone as the models. This is reportedly true of certain oxygen measuring
devices where one puts one's finger into a sensor device.18 As Liao and Huebner
see things, a technology that systematically disadvantages one group of people—
e.g., people with a dark skin tone—is in itself oppressive. If that is right, the
technology might be seen as being in itself bad. It might be seen as being in
itself bad since oppression is in itself a bad thing from a moral point of view.
This might be something that can be regarded as an example of a technology that
is non‐instrumentally negatively misaligned with human values.
4.58 Another potential example of technologies that might be judged to be non‐
instrumentally bad—and in that sense misaligned with important values—is any
form of technology that can be seen as a symbol of something that is in itself bad
or negative. Recall that in Chapter 2, we saw that Kathleen Richardson argues
that sex robots symbolize negative stereotypes about sex partners, in particular
women. According to Richardson, sex robots objectify women in how they
portray women. If Richardson is right about this, the sex robots she is talking
about might be seen as potentially being in themselves bad if we think of
symbolizing something bad as being in itself a bad thing. We will see later on, in
Chapters 8 and 9, that some have responded to Richardson by arguing that sex
robots do not necessarily have to symbolize something bad. They might even,
according to some authors, symbolize something good and positive, at least
under certain circumstances. But if Richardson is right that they do not, then
perhaps this is an example of a technology that is non‐instrumentally negatively
misaligned with important values.
4.59 The Australian philosopher Robert Sparrow, who is known for developing
many widely discussed technology‐skeptical arguments, offers an argument
related to sex robots that is similar to Richardson's stance. According to Sparrow,
since sex robots are incapable of consenting to sex, having sex with a sex robot
represents or symbolizes rape. Rape is in itself an attack on human dignity. So, if
a technology or the way that it operates would be a representation of rape, it
might be argued that the technology is bad in a non‐instrumental way, i.e., not
just because of its effects but somehow bad in itself. As before, there are those
who have responded that sex robots would not have to represent rape. There are
even sex robot developers who have tried to develop sex robots that behave as if
they consent to sex because these sex robot developers join Sparrow in being
opposed to non‐consensual sex. So, there are two sides to this dispute. But if
Sparrow is right, this would be another example of a technology whose
operation or use might seem to be in itself objectionable.19
4.60 The same could perhaps be said of instruments of torture, such as iron
chairs, choke pears, or breaking wheels. What such technologies are designed for
(torture) and their main use (again, torture) is something that can be seen as in
itself bad and not just instrumentally bad. So that might be another example of a
form of technology that operates in a way that is in itself bad or intrinsically
misaligned with positive values. These technologies are clearly tools, but the use
or operation of these tools might be considered bad in itself.
4.8 The Control Problem
4.61 What the previous sections have been discussing—the goal of value
alignment—is often seen as one possible solution to the so‐called AI control
problem. Simply put, the control problem is that as AI gets more and more
autonomous and more and more powerful in its capabilities, the harder it will be
to retain control over it. One possible solution has been what is sometimes called
“motivational control.” If the AI is designed to have the right “motivations,” by
being aligned with human values, then human beings will have a form of at least
indirect control over the AI. The AI will then be doing what we want it to do, in
accordance with our human values.
4.62 Another form of control that is often discussed in relation to the control
problem is “capability control.” Here the idea is to put limits on the capabilities
of AI systems in different ways, and thereby retain control over the technology.
Examples of this include always having an “off‐switch,” what is called “boxing,”
and the use of an “oracle.” The first idea—the off‐switch idea—is that the
system should be designed in such a way that it is always possible to simply pull
the plug and stop the system. The second idea—the boxing idea—is that the AI
should be “boxed in,” so to speak, in the sense of only being able to operate
within certain limits or boundaries, outside of which it is not able to act or
extend itself. The third idea—the oracle idea—is that there should be another AI
system that predicts what more advanced or more powerful AI systems might be
able to do. That way, one can stop the creation or use of the more advanced or
more powerful system before it comes into existence and gets out of control.
4.63 The common problem that all of these three approaches to capability
control are often thought to have is that they are likely to work best with more
narrow and less powerful forms of AI systems. Once the AI becomes general
and creative enough in its functioning, none of the forms of capability control
are thought to be sufficiently reliable. The AI capabilities in those powerful AI
systems are thought to be able to outsmart our human efforts to limit their
capabilities, so as to be better able to achieve whatever goals the AI systems will
have been designed to have. So, it is sometimes argued, some form of
motivational control—where the goals of the AI are made to aligned with human
values—would be a more promising approach. But how safe and secure is the
motivational control approach? And how likely is it that we will be able to
control future forms of more advanced and powerful AI?
4.64 Roman Yampolskiy is a philosophically inclined computer scientist, who
was born in what was then the Latvian part of the Soviet Union, but who since
many years has been working in the United States. Yampolskiy is the author of
numerous books and articles on AI safety and security. In some of his most
recent work, he defends the startling conclusion that AI might in principle be or
become uncontrollable. At least, we might end up being faced with a choice
whereby if we want safe AI, then we might need to give up control, and if we
want to control AI, then it might not be sufficiently safe. Let us consider
Yampolskiy's reasoning.
4.65 Yampolskiy notes, firstly, that it is standard practice in computer science to
try to show that a problem is not unsolvable before one spends too much time
and resources on it. He then goes on to argue for the alarming conclusion that the
AI control problem might be an in‐principle unsolvable problem. The “AI
control problem is the definitive challenge and the hard problem of AI safety and
security,” Yampolskiy (2020: p. 2) claims. He is thereby alluding to the idea that
explaining how subjective consciousness can emerge from organic materials—
such as our brains—is the “hard problem” of neuroscience and the philosophy of
mind. So, what is the argument?
4.66 Yampolskiy thinks that capability control is at best a temporary measure,
but that motivation control is also likely to end up being an approach that leads
to our losing control over the AI systems we create—at least if we wish that
these systems remain safe for human beings. To illustrate this, Yampolskiy
draws some more distinctions among different kinds of control, using the
example of a self‐driving car with very powerful AI, and a wish on the part of a
human passenger in that car to stop the car. Supposing that the human issues the
command “please stop the car!”, there are at least four options that Yampolskiy
thinks that we should consider:
Explicit control: the AI in the car immediately stops the car, even if it is driving
at high speed in the middle of a highway, thereby interpreting the human
command literally—a very unsafe solution!
Implicit control: the AI in the car attempts to comply with the command in a
safe way, perhaps by waiting until there is a safe place to stop, such as by a
roadside gas station.
Aligned control: the AI in the car tries to interpret the human being's reasoning
behind their command—e.g., interpreting the human being as wanting to use the
restroom or buy something to eat—and again stops whenever there is a gas
station but not before then.
Delegated control: the AI in the car does not wait until the human being issues
any commands, but instead takes its own initiatives to try to do whatever it
judges would be best for the human. Perhaps instead of stopping at a gas station
where one can buy only unhealthy food, the car drives on until there is a health
store, or perhaps a gym where the human being can do a workout before eating
anything.
In discussing these options, the first thing Yampolskiy does is to comment that
retaining explicit human control over AI systems—such as in the example with
the car above—is likely to generate unsafe forms of AI. Human beings make ill‐
advised decisions every now and then. And if a powerful AI system is there to
carry out all of our explicit commands, this is likely to often be unsafe, such as if
the AI system in the car above immediately stops when a human asks it to stop.
If, on the contrary, we transfer over decision‐making about what it is safe or wise
to do to the AI system, then we seem to hand over control to it, at the very least
losing a direct and explicit form of control.
4.67 At the same time, it can of course also be that a not‐directly controlled AI
system might also behave in a way that is highly unsafe for human beings. This
means, according to Yamposkiy, that AI systems might be unsafe because they
are directly controlled by human beings or unsafe because they are not under the
direct control of human beings. So, no altogether desirable solution to the control
problem might be possible, Yampolskiy argues.
4.68 Explicit control will work—in the sense that we do retain control—in the
kinds of cases where we issue commands that AI systems could follow. But
human beings might issue contradictory commands or otherwise in‐principle
unfollowable commands. Human beings might wish for the AI system to do two
things that cannot be done at the same time. Or we might ask the system to do
something paradoxical. Yampolskiy illustrates the latter idea with the command
“disobey me!” If the AI system ignores this command, it both complies and fails
to comply with the command. And if the AI system tries to follow the command,
it will fail to comply with it by trying to comply with it. That is perhaps a silly
example. But the main point—that we might sometimes issue commands that
could not possibly be carried out—does seem to hold true.
4.69 At the other extreme is what Yampolskiy calls “delegated control.”
Yampolskiy compares this with what the AI researcher Eliezer Yudkowsky calls
the goal of fulfilling human beings' “coherent extrapolated volition”—defined as
the set of values human beings would have after a very long and idealized
process of achieving some form of collective reflective equilibrium. That is, the
AI system would implement the wishes we would have if “we knew more,
thought faster, were more the people we wished we were, had grown up farther
together” and so on. As Yampolskiy sees things, such a paternalistic AI system
that would try to help us in accordance with what it thinks are the wishes of our
more enlightened selves would not be under our control. We would, instead, be
like small children under the control of parents who think they know best what
their young children want. Delegated control, then, would be a total loss of
control.
4.70 What about the two options in between explicit control and delegated
control: implicit control and aligned control? In both of those cases, Yampolskiy
argues that there is a safety/control trade‐off that will guarantee neither the
achievement of control nor of safety. Getting more safety is likely to mean
giving up some control, according to Yamposkiy, and striving to maintain direct
control is likely to undermine safety.20
4.71 Other researchers, it is important to note, are somewhat more optimistic
about achieving advanced AI that is both safe and under a sufficient amount of
“meaningful human control,” as it is sometimes put. Authors who are optimistic
about this tend to focus, not on direct and explicit control, but rather on certain
forms of indirect control. That is why they use the phrase “meaningful human
control.” It is meant to indicate that there is some form of control, which is
sufficient to qualify as a form of control and which would also be sufficiently
valuable.21
4.9 Control as a Value: Instrumental or Non‐
Instrumental? And Are There Some
Technologies It Might Be Wrong to Try to
Control?
4.72 When AI researchers such as Stuart Russell and Roman Yampolskiy—or
philosophers discussing AI, such as Nick Bostrom—talk about AI and worry
about the above‐described control problem, their discussions typically treat
control as something that is of value. Their discussions portray control as
something valuable that we do not want to lose. It would be bad, in other words,
if we lost control over the AI systems we are creating. And it would be good if
we manage to retain control over these AI systems.
4.73 This seems hard to deny, because surely losing control is bad and retaining
control is good. But what kind of value does control have? And are all kinds of
control equally valuable? Are there perhaps even forms of control that are not
valuable? These are such fascinating questions that we will devote most of the
next chapter to them. But let us close this chapter by briefly relating this
question about the value of control back to the discussion of the value alignment
issue that was discussed for most of this chapter before we started discussing the
control problem in the previous section.
4.74 If we assume—as it makes sense to do!—that some forms of control are
good, then how can we compare the value of control with some of the other
things of value that we discussed above? One question here is whether the value
of control is primarily of an instrumental kind or whether it can also, at least
sometimes, be of a non‐instrumental kind, where having control is somehow
seen as valuable in itself, and not only as a means to other ends. As was just
noted, we will discuss this—and other related questions—in the next chapter.
But let us here make the following observation.
4.75 Suppose for the moment—and this will be a supposition we will challenge
in later chapters!—that there can be technologies that are good in themselves or
technologies whose operation is good in itself. Potential examples, we saw
above, include technologies that could be our friends, our romantic partners, or
that could be moral agents or moral patients. As noted above, friends and
friendship, and also romantic partners and romantic relationships, are usually
considered to be good and valuable in themselves, and not only as means to
other ends. Likewise, when moral agents act in morally good ways and are,
thereby, being virtuous, this is also sometimes seen as good and worthy in itself,
and not only as a means to other ends. Lastly, to give just one more example: if
somebody (e.g., a human being or some animal) is seen as having a moral status,
then that person or animal is seen as important or valuable in themselves, and
not only as a means to other ends. To repeat the Kantian principle that many
people agree with: we should always treat each person as an end in itself, and
never as a means only.
4.76 Now, notice the following thing: it is usually considered very problematic,
if not outright wrong, to try to control our friends, our romantic partners, or other
moral agents or moral patients. Somebody who tries to control another person
can appear to act in an immoral way, treating the other person as a tool or a
slave, i.e., something they can control and use for their own purposes, as they
please. Likewise, while we can ask our friends and romantic partners if they are
willing to help us with things—and our friends and romantic partners might be
under a certain amount of pressure to be willing to help us out if they are to
count as good friends or romantic partners—we should not simply count on their
help. Rather, we need to ask if they are willing to do certain things for us.
4.77 In other words, it is wrong and bad to try to control our friends and
romantic partners. More generally, we are supposed to let other moral agents
make their own decisions and not try to control their decisions or lives more
generally. This is an important part of what it is to respect other people as
persons in their own right.
4.78 This seems to mean that if philosophers like John Danaher, Helen Ryland,
Janina Loh, and so on are right that technologies like robots or AI systems can
be our friends, romantic partners, or moral agents or patients, it might be wrong
to try to control these technologies. After all, it is, as we just said, wrong to try to
control our friends or romantic partners or other people more generally. So, we
seem to have one class of potential technologies that it would be at least
problematic to want to try to control: namely, those that would play the kind of
roles that some philosophers and others envision that some technologies might
come to play. If a robot, say, is supposed to be our friend, romantic partner, or
perhaps even our moral equal, then a new control problem arises: it might
become wrong or morally bad to try to have complete control over that
technology.
4.79 It appears, then, that only technologies that are fit to be considered as mere
tools or instruments are ones that it seems always okay, unproblematic, and
perhaps also unconditionally good to try to control. If technologies can ever
become more than tools—if they can become our friends, romantic partners, or
moral agents or moral patients—then suddenly the desire to control technologies
potentially becomes morally problematic. We here seem to have another reason
to reflect seriously on what we called Joanna Bryson's normative instrumental
theory of technology above. Bryson, recall, argued that we should not create any
technologies that we think it would be wrong to treat as anything but mere tools.
We should only, in other words, create technologies that can be mere means and
that are not ends‐in‐themselves. The fact that we want to control technologies
and we think that it is, in general, good to control technologies can potentially be
seen as a reason in favor of a Bryson‐inspired normative instrumental theory of
technology.
4.80 Perhaps it is possible to create robots and other technologies that would be
more than tools—e.g. that could be our friends, and so on—but if we want to
remain in complete control over the technologies we create, then we might have
moral reasons to avoid creating such technologies. Or, alternatively, we have to
give up on our desire to control technologies. Just like Yampolskiy argues that
we cannot have both fully controlled technologies and completely safe
technologies, it might be argued that we cannot both have fully controlled
technologies and technologies that are supposed to be our friends, romantic
partners, or moral equals.
4.81 At this point, it might be suggested that this is a silly problem: it might be
argued that Danaher, Ryland, and others are wrong that robots could be our
friends or romantic partners, or that any AI or other technologies could ever be
moral agents or moral patients that would be our equals in any way. So, there is
no problem with trying to retain complete control over all technologies we
create. Later we will discuss those issues. That is, we will discuss whether
technologies could be our friends, romantic partners, moral agents and patients,
and so on. But for now, let us stay with the fascinating topic of control a little
longer.
4.82 In the next chapter, we will move on to thinking about control and its value
in the context of what are sometimes called behavior change technologies. The
question will be: if technologies are created that are meant to steer, nudge,
influence, or manipulate how people behave, does that mean that we might lose
control over ourselves in a way that is bad? Could this work as a moral argument
against these so‐called behavior change technologies? Or might perhaps such
technologies ultimately give us greater control over ourselves, which might be
considered as being something positive?
Annotated Bibliography
Nick Bostrom (2014). Superintelligence: Paths, Dangers, Strategies, Oxford:
Oxford University Press.
A much‐discussed book about the control problem, which formulates worries
about super‐intelligent AI systems from a philosophical point of view.
Iason Gabriel (2020). “Artificial Intelligence, Values, and Alignment,” Minds
and Machines, vol. 30, no. 3, pp. 411–437.
An explanation of the value alignment problem, along with an interesting and
informative discussion of the question of whose values AI technologies
should be aligned with.
Katleen Gabriels (2021). Conscientious AI: Machines Learning Morals,
Brussels: VUB Press.
An informative and engaging book about different ways in which
technologies might clash with human values and what to do about that.
Shen‐Yi Liao and Bryce Heubner (2021). “Oppressive Things,” Philosophy and
Phenomenological Research, vol. 103, no. 1, pp. 92–113.
The authors argue that technologies can have oppressive qualities or effects.
Stewart Russell (2019). Human Compatible: Artificial Intelligence and the
Problem of Control, London: Penguin.
An accessible discussion of the control problem by one of the world's leading
AI researchers.
Stewart Russell and Peter Norvig (2021). Artificial Intelligence: A Modern
Approach 4th edition, Hoboken: Prentice Hall.
The most widely‐used textbook on AI, which contains, among many other
things, an interesting discussion of how to understand what AI is.
Aimee Van Wynsberghe (2021). “Sustainable AI: AI for Sustainability and the
Sustainability of AI,” AI and Ethics, vol. 1, no. 3, pp. 213–218.
A plea to consider both whether AI could be used to promote sustainability
and, more importantly, to reflect critically on whether AI technologies are
sustainable from an environmental point of view.
Carissa Véliz (2020). Privacy is Power: Why and How You Should Take Back
Control of Your Data, London: Penguin.
A powerful discussion of risks and ethical issues related to data‐collecting
technologies.
Roman Yampolskiy (2020). “On Controllability of AI,” http://arXiv.org,
https://arxiv.org/abs/2008.04071.
A fascinating discussion, by a leading computer scientist working on AI
safety and security, about whether AI is in principle controllable or in
principle uncontrollable.
Notes
1 For more information, see the Wikipedia article “Stanislav Petrov”:
https://en.wikipedia.org/wiki/Stanislav_Petrov
2 Listen to Bostrom on superintelligence on this episode of the “Philosophy 247”
podcast: “Super Intelligence”: https://philosophy247.org/podcasts/super‐
intelligence
3 Sidney Fussell (2017) “Why Can't This Soap Dispenser Identify Dark Skin?”
Gizmodo, https://gizmodo.com/why‐cant‐this‐soap‐dispenser‐identify‐dark‐
skin‐ 1797931773
4 For more information, see the Wikipedia entry “Dartmouth Workshop”:
https://en.wikipedia.org/wiki/Dartmouth_workshop
5 A short video about Alan Turing from Cambridge University: “Alan Turing—
Celebrating the Life of a Genius”: https://www.youtube.com/watch?
v=gtRLmL70TH0
6 You can hear Gabriel discuss value alignment on this episode of the “Future of
Life Institute” podcast: “Iason Gabriel on Foundational Philosophical
Questions in AI Alignment”: https://futureoflife.org/2020/09/03/iason‐
gabriel‐on‐foundational‐ philosophical‐questions‐in‐ai‐alignment
7 Korsgaard discusses her ideas on this “Philosophy Bites” episode: “Christine
Korsgaard on the Status of Animals”:
https://philosophybites.com/2015/02/christine‐korsgaard‐on‐the‐status‐of‐
animals.html
8 To learn more about the ideas in Urmson's TED talk, you can watch the video
“How a Driverless Car Sees the Road”:
https://www.ted.com/talks/chris_urmson_
how_a_driverless_car_sees_the_road
9 See the editorial “Driverless Cars are Stuck in a Jam,” originally published in
2019 and updated in 2022, in The Economist:
https://www.economist.com/leaders/ 2019/10/10/driverless‐cars‐are‐stuck‐in‐
a‐jam
10 Anybody who wishes to become slightly paranoid about using data‐collecting
technologies should read Véliz's book Privacy Is Power: Why and How You
Should Take Back Control Over Your Data. Véliz summarizes her overall
argument here: Carissa Véliz (2019): “Privacy Is Power: Don't Just Give
Away Your Privacy to the Likes of Google and Facebook—Protect It, or You
Disempower Us All,” Aeon: https://aeon.co/essays/privacy‐matters‐because‐
it‐empowers‐us‐all
11 See Katleen Gabriel's video presentation “We Design Technology,
Technology Designs Us”: https://www.youtube.com/watch?
v=6nTkWPOlwUk
12 You can hear Russell discuss the King Midas problem and related issues on
this episode of the “80,000 Hours” podcast: “#80: Professor Stuart Russell on
Why Our Approach to AI is Broken and How to Fix It”:
https://80000hours.org/podcast/episodes/stuart‐russell‐human‐compatible‐ai
13 You can listen to Smith and Danaher on friendship with robots in this episode
of the “Dolores Project” podcast: “Family, Friendship and Robot Philosophy
w/John Danaher”: https://anchor.fm/joshua‐k‐smith6/episodes/Family‐‐
Friendship‐‐and‐ Robot‐Philosophy‐w‐John‐Danaher‐e160fkh
14 See, for example, this article: Julie Beck (2013) “Married to a Doll: Why One
Man Advocates Synthetic Love,” The Atlantic:
https://www.theatlantic.com/health/archive/2013/09/married‐to‐a‐doll‐why‐
one‐man‐advocates‐synthetic‐ love/279361
15 For more references, see the annotated bibliography of and notes to Chapter
9.
16 See Chapter 7 for relevant references.
17 See Chapter 8 for the relevant references.
18 See Vanessa Carbonell & Shen‐Yi Liao (2021) “Some Medical Devices Don't
Mean to be Racist, but They are,” Psyche: https://psyche.co/ideas/some‐
medical‐devices‐ dont‐mean‐to‐be‐racist‐but‐they‐are
19 For references, see Chapter 8.
20 You can hear Yampolskiy discuss these ideas on this episode of the “Future of
Life Institute” podcast: “Roman Yampolskiy on the Uncontrollability,
Incomprehensibility and Unexplainability of AI”:
https://futureoflife.org/2021/03/19/roman‐yampolskiy‐ on‐the‐
uncontrollability‐incomprehensibility‐and‐unexplainability‐of‐ai
21 A video about the idea of meaningful human control over self‐driving cars
from a Dutch research project: “Who is in Control of Self‐Driving Cars?”
https://www.youtube.com/watch?v=nPB2IeY9u9Y
5
BEHAVIOR CHANGE TECHNOLOGIES,
GAMIFICATION, PERSONAL AUTONOMY, AND
THE VALUE OF CONTROL
5.1 A Better You?
5.1 In a 2018 video commercial for a smartwatch called “the Apple Watch,” the
first thing the viewer sees is a very laid‐back man sitting on a couch, drinking
coffee. The man then gets a surprised look on his face. He glances over and sees
another version of himself, who is looking down at a smartwatch. The next
frame zooms in on the watch. The watch displays the message “time to stand!”
Both versions of the man then immediately stand up. Next, we see them walking
down the street. Several other versions of this same man are now seen using the
smartwatch in all sorts of ways: to make calls, to keep track of their exercising
(one jogging version of the man looks down at a message that reads “behind
target!”), and so on.
5.2 Toward the end of the clip, we see the by far sportiest version of the man.
The other versions of him are wearing a casual t‐shirt and some sweatpants. But
this highly athletic version of the man is sporting swimming trunks and
swimming goggles. He enthusiastically runs down to the ocean, jumps in, and
starts swimming, looking very much like a man of action. This is a big contrast
to the couch‐potato version of this man we saw at the beginning of the clip. The
commercial then draws to a close. A text appears on the screen that reads as
follows: “there's a better you in you.”1 The intended message is clear: if you start
using this smartwatch, you will become fitter. You will start achieving all of your
goals. And you will gain increased control over yourself and thereby become a
“better you.”
5.3 This smartwatch commercial nicely illustrates what many people who
voluntarily use what are sometimes called “behavior change technologies” want
to achieve. They want to achieve a greater degree of control over themselves and
their lives. Behavior change technologies are, roughly, technologies used to
influence people's motivation and behavior. They influence people by tracking
and logging data, giving them information and statistics, gamifying certain
activities, redirecting their attention in certain directions, giving them prompts,
notifications, or reminders, and so on.
5.4 Such technologies are sometimes used for “self‐tracking.” This is the idea
that we can live better lives and achieve greater control over ourselves and our
goals if we quantify, track, log, run statistics on, or in other ways make use of
data about ourselves of the sorts that can be collected by different forms of
technologies. For example, people use apps to track their eating, their exercise,
their menstrual cycle, their sex life, their sleeping patterns, and all sorts of
things.2
5.5 We will here be concerned with behavior change technologies more
generally. But the idea of self‐tracking as one potential use of these technologies
is one of the things that we will be keeping in mind during the discussion in this
chapter. We will also be keeping the idea of gamification in mind. Roughly, this
is the idea that one effective way of influencing people is to introduce game‐like
elements into activities that are normally not like games. This is one key strategy
often used to change or influence people's behavior.
5.6 On one hand, behavior change technologies hold out the promise of giving
us greater control over ourselves. On the other hand, however, these technologies
also frequently give rise to worries about a potential loss of control. For
example, when we use smartwatches, smartphones, and other technologies that
track what we do and log various kinds of personal data about us, there is a
commonly raised worry that this data will be used for purposes outside of our
control by the companies that offer the products and services associated with
these technologies.
5.7 In relation to this worry, it is sometimes said that “if it is free, then you are
the product.” What lies behind this expression is the observation that companies
will collect large amounts of information about you, and then monetize that data,
e.g., by selling advertising space to other companies and organizations, or in
other ways use it, in order to profit from the apps that you get to use “for free.”
Information about us—which we may not realize is being collected—may be
used by many different organizations to try to influence our behavior. This is
done to a greater extent than many people realize. It is the business model
behind many services that we get to use for “free.”3
5.8 In other words, in addition to the behavior change technologies that we
ourselves voluntarily choose to use, there are also behavior change technologies
that others expose us to—such as different forms of technological nudges,
externally imposed gamification, and so on. This is sometimes labeled
“persuasive technologies.”4 Such technologies can give rise to worries about
being under other people's control. In general, we do not like the idea of others
manipulating and influencing us to do what they think we should be doing. This
is often viewed as a threat to personal autonomy.
5.9 In general, then, behavior change technologies seem to be a double‐edged
sword when it comes to the control we want to have over ourselves and the
personal autonomy we value in our lives. While we may use some behavior
change technologies because we think it will give us more control over
ourselves, we also worry that such technologies may give others more control
over us.
5.10 In the previous chapter, we saw that artificial intelligence also gives rise to
concerns about control. The control problem about AI is the problem of how to
maintain control over increasingly complex and powerful AI systems. In that
discussion, it is usually taken for granted that a loss of control is a bad thing. In
the next chapter we will also see that control is a key issue in discussions about
moral responsibility and technology. In other words, the topic of control and its
importance runs like a thread through a lot of technology ethics. So, it is a good
idea to zoom out a little and reflect on what exactly we understand by control
and why it is so important to us.
5.11 The most general question this chapter discusses is therefore this: What is
control, and why does it matter? Is it instrumentally valuable only? Or is it
valuable in some non‐instrumental way as well, as an end in itself? The overall
aim of this chapter is twofold: first, to discuss some key features of what it is
that we care about when we value control, and second, to reflect on how we
should understand the value of control—for example, how it relates to the ideal
of personal autonomy. The discussion in this chapter will primarily be framed
around behavior change technologies. But as just noted, the issues that we
consider in this chapter are of great relevance also for artificial intelligence, as
well as for technology ethics more generally.
5.2 Behavior Change Technologies and
Gamification
5.11 To motivate the approach to control and its value that we will get to below,
it may be useful to first reflect a little more on what we should understand by the
phrase “behavior change technologies.” As we do so, we can first recall that
some philosophers of technology—particularly those associated with the so‐
called post‐phenomenological school of thought—have argued convincingly that
most technologies have different kinds of effects on human behavior.
5.12 As we saw in Chapter 1, for example, the Dutch philosopher Peter‐Paul
Verbeek, discusses two ways in which technologies tend to “mediate” human
life. First, technologies have an impact on how we experience and perceive the
world. Second, technologies have an impact on what we are able to do and how
we are able to do it. So, if we think about technology in this way, it might seem
hard to make any distinction between behavior change technologies, on the one
hand, and other technologies, on the other, since most technologies have an
impact on human behavior.
5.13 Nevertheless, when technology ethics researchers talk about “behavior
change technologies,” they usually intend to refer to a certain subset of the
technologies available to us. They are usually particularly interested in
technologies that are developed to influence patterns of behavior where it is hard
to change these patterns simply by deciding to do so.
5.14 Behavior change technologies can be self‐directed or other‐directed in their
use. That is, we may choose to use them because we want to change certain
patterns in our own behavior. Alternatively, others might use these technologies
to influence certain patterns in our behavior. Moreover, one and the same piece
of technology might serve as a means for furthering different people's aims. To
take the case of the smartwatch in the introduction: the user might want to
change their exercise patterns, but the company behind the technology might
want the technology to influence the user's consumer behaviors as well.
5.15 A person might wish to change her own patterns of behavior in some way,
and they might be able to change particular instances of behavior by sheer
willpower. But changing the general pattern may be harder to do. Somebody
might wish, for example, to exercise more, in a more consistent and systematic
way. This might lead them to go running on a particular occasion. But making
this into a habit might be easier if they have an app that involves some
gamification of the sort where the person can compare her own running with that
of others on a leaderboard. The running app can also help to keep track of how
long each run is, what speed the person is running at, and so on. This can serve
to further boost the user's motivation to exercise more consistently. It can
become easier to keep track of one's progress, which can be motivating.
5.16 Let us next consider an example of a behavior change technology directed
at other people's behavior. If a car manufacturer wishes to make sure that
everyone in a car is using seat belts, for instance, they can of course simply ask
them to do this. But it might be much more effective to design the car in such a
way that it makes an annoying sound whenever the people in the car are not
wearing a seatbelt. This is a good example of a so‐called persuasive technology.
You yourself might choose to buy a car that will give you different kinds of
reminders—and that will annoy you with signals or beeps if you do not act in
certain ways—e.g., driving in an energy‐efficient and environmentally friendly
way. Again, the distinction between behavior change technologies we voluntarily
choose to use and behavior change technologies others “impose” on us is not
always clear‐cut.
5.17 In general, then, when we look at behavior change technologies in this
chapter, we will be particularly interested in technologies aimed at influencing
habits or patterns of behavior that are hard for people to directly control in a
consistent way. In terms of what technological solutions are available, these
include things such as reminders, notifications, signals (which might be
annoying signals), gamification, tracking or logging of information, and so on—
anything that might assist people when simply deciding to change some habit is
not likely to be sufficient to change their patterns of behavior. If we understand
the idea of behavior change technologies in this way, we have a subset that is
smaller than the whole set of all technologies, even if we agree with post‐
phenomenologists that most technologies have effects on human behavior.
5.18 One more thing before we get to the issue of how to understand control: the
term “gamification” has been used a few times in this chapter, including in this
section a few paragraphs up. That is not an everyday term. So, while we already
briefly defined it at the end of Chapter 1 and in the introduction earlier, let us
again briefly reflect on what this expression refers to. Simply put, some activity
or part of human life is gamified when game‐like elements are introduced into it.
This is often done in order to increase people's motivation to perform the activity
by giving them an external motivation. The activities that are “gamified” may
themselves not be game‐like to begin with.5
5.19 If we, for example, start giving people points, or we start ranking them in
terms of how they compare to others, these are examples of ways of gamifying
activities. This can be voluntary and fun: e.g., gamifying one's exercise by using
an app that creates a ranking of who does the most running. This is something
that the author of this book does together with his wife and family‐in‐law. We
happen to enjoy running without the need to gamify the activity. However, the
friendly competition adds an extra amount of motivation to run more and run
longer. This is an example of gamification that is welcomed by those whose
activities are gamified. In some cases, however, gamification may also be
nonvoluntary, imposed by others, and not necessarily welcome.
5.20 An employer, for example, might gamify the work the employees do, by
collecting data about their performance and then ranking them, so that all
employees can see who works the most and the least. In one news story from
2011 in Forbes magazine, for example, it was reported that employees at
Disneyland complained that their employer was using an “electronic whip” when
their performance was displayed and ranked on a leaderboard in the staff room.
This did motivate people to work harder, because they did not like the idea of
being ranked low, but at the same time they did not like this ranking.6
5.21 Similarly, some academic researchers complain that their research outputs
are gamified in problematic ways when universities use metric data about
numbers of publications and citations and so on to rank researchers in terms of
their output. What does all these have got to do with control and personal
autonomy? Some people worry that when their activities are gamified by
somebody else, who introduces some sort of game‐like motivation into some
activity, then there is a certain amount of manipulation of our motivation.
5.22 Many people do not like being led to engage in activities partly for other
reasons (e.g., a sense of competition) than those that we may prefer to act on the
basis of (e.g., an intrinsic desire to perform the activity in question). For
instance, academic researchers may prefer being motivated by their intrinsic
interest in their research subject. And they may not like it when others try to
motivate them by turning academic publishing into a form of competitive game.
5.23 Gamification, then, is another thing that is a double‐edged sword. Turning
something into a game can sometimes make it more fun. But it can also take the
fun out of something. And it can change the nature of the activity, sometimes for
the worse. This is something we will also discuss again in Chapter 9, when we
briefly discuss the idea of gamification in the context of romantic and other
kinds of personal relationships.
5.3 Control: Three Basic Observations
5.24 Let us now turn to the issue of what it is that we value or care about in
valuing control. Control is one of those things people are concerned with in most
parts of life. We discussed control worries in relation to AI in the previous
chapter and we are discussing control worries in relation to behavior change
technologies in this chapter. But most people care about control more generally.
For example, you might want to have control over how much money you are
spending, how long what you are cooking is in the oven, or over your bodily
functions. You might want to have control over your violin playing, your
vegetable garden, or the word count in an essay you are writing.
5.25 Since it is something we care about in most areas of life, a lot can be said
about control. In approaching the question of what we should understand by
control in relation to technologies in particular, we can start by making three
general observations.
5.26 It can firstly be observed that the idea of control is not a simple notion.
Rather, it is best understood as a multidimensional phenomenon with many
different aspects to it. For example, one dimension of control is the ability to
monitor what we are controlling. Another might be the capacity to steer or
otherwise influence what we are controlling. And yet another dimension could
be whether we have a proper understanding of what it is we are trying to control.
5.27 In different cases, different aspects of control might be present, to different
degrees. For example, you might not be able to directly steer some process. But
you might have a good understanding of it. And it might be strongly responsive
to your preferences. This might happen if somebody is acting on your behalf in
some way. Or you might be able to directly steer something, such as some
technology (e.g., a car or the movements of a cursor on a computer screen). But
you might not have a good understanding of how the technology works.
5.28 In cases of what might be called “maximal control,” all the different aspects
of control we can imagine are maximally present. In such cases, the persons in
control have a maximal degree of control along all the different dimensions of
control that there are. But more commonly, while some dimensions of control
might be present to a significant degree, other aspects might be weaker or less
present. And different people may differ in their control‐profile, so to speak. As
we might think of it, then, control has aspects that are a little like the dials or
settings on an equalizer, where you can turn knobs or dials in different ways and
thereby get different control settings, as we might put it.
5.29 A second general idea is that control can be more or less robust. You might
have control over something (a behavior, a technology, an animal, or whatever)
only in some contexts or only in certain kinds of situations. Or you might have
control over the thing in question across a very wide range of scenarios and
contingencies. That would then be a more robust form of control.
5.30 Self‐control with respect to how much alcohol one drinks might be a good
example to illustrate this point. There are some people who are good at
controlling how much they drink if they are on their own or among people who
do not drink much. But if these people are among others who drink a lot, they
too will drink a lot, and have great difficulty in controlling their drinking. That
would be an example of people whose control over how much they drink is not a
very robust form of control.
5.31 Or, to use a more technological example: somebody who is a very good
driver might be able to control their car in many different weather conditions,
and independently of whether they are tired or energized. Somebody who is a
less good and less experienced driver may only be able to control their car in a
narrower range of circumstances: e.g., only in favorable weather conditions and
when they are not tired. The former driver has more robust control over their car
than the latter does. The general point here is that different kinds of control can
differ in their robustness.
5.32 A third basic observation is that control can be more or less direct. When
we are interested in control over some forms of technologies—e.g., advanced AI
—we may primarily be interested in indirect forms of control. The same goes for
behavior change technologies. The reason for this is that behavior change
technologies are often most needed in cases in which we lack a robust form of
direct control over certain patterns of behavior. That is to say, we cannot easily
just decide to change the patterns of behavior in question; a simple act of will is
not sufficient. We cannot simply steer ourselves or others, like we might steer a
car by turning the steering wheel. So, the forms of control we could achieve by
using behavior change technologies are typically a more indirect form of control.
5.33 Alternatively, we might be worrying that other people are achieving indirect
control over us by creating behavior change technologies that we are exposed to.
As noted earlier, we are here interested both in opportunities for greater control
created by behavior change technologies as well as in potential threats to control
created by behavior change technologies. In both kinds of cases, the most
relevant kinds of control will often be indirect in nature.
5.4 Key Dimensions of Control Discussed in
Different Areas of Philosophy
5.34 In order to create an overview of the different aspects we can associate with
different kinds of more or less robust indirect control, it is useful to look at
discussions of control in different areas of philosophy. It is useful to do this
because philosophical theories in different domains may be concerned with more
or less direct forms of control. Moreover, they may also be concerned with
different dimensions of control.
5.35 In particular, we will briefly consider some ideas about control from
political philosophy, technology ethics, and the theory of self‐control. There will
be some overlap in the ideas we consider below. But there will also be some
differences in what is highlighted as particularly important in the parts of
philosophy in question.
5.36 We can start with some things that the influential Irish philosopher Philip
Pettit suggests about the nature of control when he discusses control in the
context of political philosophy. Pettit is interested in the kind(s) of control
citizens can have in democratic states. He is also interested in how that form of
control is (and is not!) related to consent. In general, Pettit suggests that control
has two essential components, which he calls “influence” and “direction.”
5.37 The first component is that we should have a certain degree of influence
over what we are trying to control. But just any old influence will not do. The
influence should have a direction that fits with what we desire to achieve. To
illustrate this point, Pettit imagines somebody who is pretending to be a traffic
cop. The person is standing in the middle of the road and is making the sorts of
arm gestures that traffic cops make. This person might have an “influence” on
what drivers do. But the person might not be able to influence people's driving
according to any pattern the person wishes to achieve. So, the control dimension
that Pettit calls “direction” is missing.
5.38 Using an example from the well‐known economist and philosopher
Amartya Sen, Pettit notes that the sort of control he talks about with the two just‐
mentioned dimensions can be indirect and delayed in time. A patient in a coma,
in Sen's example, may not have any direct control over what the medical staff is
doing. But there might be an advance directive in place that enables the coma
patient to be treated in accordance with her wishes. This patient has an influence
over the direction of her care, because of this advance directive. The patient is
therefore in control—Pettit suggests—over what happens to her, even though she
is in a coma. But this is an indirect form of control. The patient is not aware of
what is happening. She is unable to directly communicate any real‐time
instructions to the medical staff caring for her.
5.39 We can also briefly consider something Pettit says about the relation
between consent and control. This is relevant to behavior change technologies
and many more digital technologies more generally. They typically involve the
giving of consent. Notably, Pettit claims that consent and control do not always
go hand in hand. The extreme case Pettit uses to illustrate this is a case of a
person who sells themselves into slavery. Such a person might consent to
becoming a slave, but loses control as a result of this. The general point here is
that having consented to something is no guarantee to being in control over it.
5.40 This last point is worth mentioning in this context. Many digital
technologies, including behavior change technologies—e.g., apps on people's
smartphones—only start working after one gives one's consent by clicking
“agree” after having been shown a very complicated consent form. Importantly,
this does not do much to guarantee that we will have very much control over
what the behavior change technologies might do, e.g., collect data and so on.
There might instead be an illusion of control.
5.41 Let us now consider a recent discussion of control in our own field of
technology ethics. Specifically, we will consider some claims about indirect
control that the technology ethicists Filippo Santoni de Sio and Jeroen van den
Hoven make in the context of discussing what they call “meaningful human
control” over artificially intelligent automated systems. What they mean by
“automated systems” are things like self‐driving cars or military robots, both of
which have some degree of so‐called functional autonomy. That is, in certain
circumstances, these technologies can operate on their own, without direct
human steering or active controlling. Accordingly, the question arises of how we
can retain meaningful human control over such technologies, especially since
they sometimes create significant risks. For instance, self‐driving cars might
crash into people, as we discussed in Chapter 3. Military robots might be used to
kill people, potentially selecting their own targets. So, what kind of indirect, yet
meaningful, control can we have over such technologies?
5.42 Santoni de Sio and Van den Hoven suggest a two‐factor theory with a
“tracking” condition and a “tracing” condition. According to the tracking
condition, one dimension of meaningful human control concerns whether the
technology behaves in a way that tracks morally relevant reasons human beings
have to want the technologies to behave in certain ways. A self‐driving car, for
example, might behave in a way that fits with the user's morally relevant reasons
to want to arrive at a certain destination in a safe way, which does not involve
harming other people. If a self‐driving car conforms its behavior in a sufficiently
reliable way to these reasons, one key aspect of meaningful human control has
been achieved, according to this “track” and “trace” theory. (What Santoni de
Sio and Van den Hoven refer to as “tracking,” we can note here, seems to be
very similar to what many AI safety researchers refer to with the expression
“value alignment.”)
5.43 The tracing condition, in turn, is that it should be possible to trace the
behavior of the technology back to some person or persons with an
understanding both of how the technology works and of the moral significance
of having a technology like this operating in society. Such persons can sensibly
be held responsible for the behavior of the technology, according to Santoni de
Sio and Van den Hoven. This is another thing that can help to make it the case
that there exists a certain degree of meaningful human control over the
technology in question. On this view, then, people can have an indirect form of
“meaningful” control over technologies if (1) those technologies behave in ways
that we have morally relevant reasons to want them to behave in and (2) there
are people who understand the functioning and moral significance of these
technologies.
5.44 Consider lastly another domain of philosophical theorizing about control:
namely, the philosophy of self‐control. In particular, let us consider what the
German philosopher Dorothea Debus says about people's ability to regulate
aspects of their mental lives over which they lack direct voluntary control, such
as moods or other ways in which we feel. Debus notes that it might be possible
for us to, in certain ways, guide how we feel or what our moods are in a goal‐
directed way, even if we cannot directly control this (the “guidance condition”).
There might be interventions we can take that help us to exercise such guidance
(the “intervention condition”). Such interventions might be more or less effective
(the “effectiveness condition”). And we might have better or poorer
understanding over the relevant aspect of our own mental lives (the
“understanding condition”).
5.45 Take the issue of whether we are in a good mood, for example. We often
cannot directly control whether we are in a good or bad mood simply by
deciding to be in a good or bad mood. However, if we know that we like things
such as jazz or hot chocolate, we know that there is an intervention we can take
(e.g., listening to jazz or drinking hot chocolate) that is likely to result in a
change in our mood. That way, we can guide our mood in a more or less
effective way. We then have some measure of indirect control over whether our
mood is good or bad.
5.46 This whirlwind tour of discussions about control in different areas of
philosophy helps us to identify a number of different aspects we can associate
with different forms of more or less robust direct or indirect control. We do not
necessarily need to think that there is a choice we have to make among these
different ways of thinking about control. Rather, we can view the just‐reviewed
discussions as identifying or teasing out aspects of control that can be more or
less relevant in different contexts.
5.47 In summary, control can involve aspects such as:
Value (or will) alignment: does the thing you wish to control track or correspond
to your values, your will, your judgments about what is best, or what you want
or have reasons to achieve?
Interventions: are there direct or indirect interventions available to you, which
you can take to influence what you wish to control to make it conform to your
will or your values?
Understanding: do you have a clear understanding of the things you wish to
control?
Monitoring: are you able to monitor or keep track of what you want to control?
Robustness: how robustly does what you want to control conform to your values
or your will? How robust is your ability to make interventions to conform what
you want to control to your values or will? And how robust is your
understanding of, or ability to monitor, what you want to control?
These are some (though not necessarily the only) key aspects of control that
people care about when they care or worry about control. Different areas of
philosophy will focus more on some aspects and less on others. But we do not
need to conclude that some areas of philosophy are getting things more right
than others. It appears better to think of different dimensions of control as
mattering in different ways in different contexts.
5.5 Behavior Change Technologies and the
“Subjects” and “Objects” of Control
5.48 If we conceive of control as a multi‐aspect phenomenon along the lines
described earlier, we can see how technologies—including behavior change
technologies—can create both opportunities for and threats to control. In order to
discuss this, we can draw another philosophical distinction that might be useful
in this context. Namely, a distinction between what we can call the subject of
control and the object of control. By “the subject of control,” we can refer to the
person or persons who are in control of something: whoever possesses some
degree of control over something. By “the object of control,” we can refer to the
thing that somebody might have control over.
5.49 In the case of self‐control, the subject and object of control might be
thought to be the same: the self. In other words, you are controlling yourself or
some aspect of yourself. But typically, the subject and object of control will
differ from each other. For example, if you are exercising control over a certain
piece of technology, then you are the subject of the control and the technology in
question is the object of that control. If other people have some sort of power
over you, then you are the object of control, while those people are the subjects
of that control.
5.50 When it comes to behavior change technologies and the subject/object
distinction, the object of control can typically be thought to be certain behavior
patterns. At least, that is how it seems natural to think about things if one
understands behavior change technologies in the way that we did earlier. For
example, the behavioral pattern in question might be somebody's tendencies to
exercise, to take their medicine at a certain time of the day, to wear their seatbelt,
or to choose healthy food options in a cafeteria.
5.51 What about the subject of control in the case of behavior change
technologies? Here it can sometimes be the case that the main subject of control
is a person who wishes to change his or her own behavior. If you, for example,
voluntarily use some behavior change technology to make sure that you exercise
more and that your exercise has certain features (e.g., that you run for a certain
amount of time, at least a certain distance, etc.), then you might be thought to be
the subject of control.
5.52 But other people might also be the subject of control when certain behavior
change technologies influence your patterns of behavior. And this then raises the
ethical concern that your personal autonomy may be threatened. That is to say,
somebody else might employ behavior change technologies as a means of
influencing your behavioral patterns. Moreover, they might succeed in doing so,
with or without your knowledge or consent. That is, you might not even realize
that your behavior patterns are influenced by technologies others use to steer or
influence your behaviors to conform to certain patterns that they desire to
achieve.
5.53 For example, many websites and smartphone apps—including the most
commonly used social media sites—are designed to maximize user engagement
or to make users click on links, with purposes such as getting the user “hooked”
or making the website or app more attractive to companies that might want to
purchase advertising space. Or consider the notifications that most apps are
constantly sending the user, so as to draw the user's attention away from other
things and back toward engagement with the app.
5.54 These are some common examples of ways in which technologies can be
designed so as to change the habits or behavioral patterns of the user. It might be
thought that these and similar types of influencing designs can clash with the
ethical ideal of personal autonomy. For example, the designs might clash with
the ideal that we should be acting in the service of ends we choose for ourselves
and for reasons we ourselves endorse. That is how the ethics researchers Daniel
Susser, Beate Roessler, and Helen Nissenbaum put things in a discussion of
threats to personal autonomy posed by manipulative technologies.
5.55 When we reflect on who is in control when behavior change technologies
and other technologies are being utilized, we can use the different control
dimensions discussed earlier as a basis for our ethical analysis. We can ask
critical questions such as:
To whose values or will (to whose preferences, to whose judgments about what
is best, to whose wishes, etc.) are the technologies in question making your
patterns of behavior conform?
Who is able to make certain interventions with, or in relation to, the technologies
in order to influence your behavioral patterns to conform to certain ideas about
what they should be?
Who has an understanding of how the technologies might be used to impact the
patterns of behavior in question?
Who is able to monitor these patterns of behavior and the way(s) in which the
technologies influence those patterns?
In how many different types of contexts is it possible for the technologies in
question to reliably influence the patterns of behavior in the way(s) intended?
By answering such probing questions, we can locate subjects of control with
respect to technologies. That is, we can determine who is in control. We can also
get clarity on how multi‐dimensional and how robust their control is. Notably, by
asking these kinds of questions, we might sometimes also find that with respect
to different aspects of control, different people or organizations may exercise
more or less control over the same thing.
5.56 For example, there might be some person (e.g., the person whose behavior
is being influenced) who wants the relevant behavior patterns to change, and that
person might have certain interventions available to them. But that person might
not have a very good understanding of the mechanisms by which a technology is
able to influence his or her behavior patterns. Somebody else might have a better
understanding of this. Somebody else might also be able to closely monitor what
the person is doing for the reason that the technology in question (e.g., some
self‐tracking app) is collecting a lot of data that other parties may have access to.
5.57 In these ways, we might find that different aspects of control are sometimes
spread out over different people. If this happens, and everyone is on the same
team, so to speak, then we might regard that team as exercising a sort of joint
control over the behavior patterns that the technologies are targeting. For
example, consider a behavior change technology used in a medical context. It
might be that a doctor advises a patient to use a behavior change technology,
such as an app that tracks some aspect of the patient's health. Perhaps the patient
and the doctor constitute a sort of team where different team‐members score
higher or lower on the different measures of control. Most of the control might
then still be distributed within this “team.” But, alternatively, there might also be
third parties who score high on some of these measures, and who might therefore
also have some control over those behavioral patterns. The company that
developed the app might collect data, for example, which gives them an ability
to monitor the patient, perhaps without the patient and the doctor fully realizing
this.
5.58 In order for there to be a solid business case attractive to companies
creating behavior change technologies, there might need to be ways in which
different people can have a share in the different aspects of control we can
imagine in relation to the behavior change technologies in question. Moreover,
sometimes more than one pattern of behavior may be the intended object of
control. To put things in somewhat abstract terms: for one subject of control, a
certain pattern of behavior might be their object of control. For some other
subject(s) of control, another related pattern of behavior might be their object of
control—all while the same behavior change technology is in use. For example,
a person might download and use a behavior change technology in the form of
an app on their phone in order to influence their own eating or exercise patterns.
The data collected or tracked by this technology might then be used by the app
creator to sell advertisement space that targets the app‐using person with the
ultimate goal of influencing some consumer behavior of theirs.
5.59 In general, when we are thinking about control in relation to behavior
change technologies, asking the sorts of critical questions formulated earlier
appears to be a good method for locating the distribution of the different aspects
of control. It is also a good method for assessing how direct or indirect, and just
how robust, that control is. Sometimes we will find that the control factors are
distributed narrowly. But more often we will find them distributed over different
people, who may or may not be “on the same team” in the sense of having
shared aims and a shared understanding of those aims.
5.6 The Value and Ethical Importance of Control
5.60 Earlier, we have been considering what control is. Among other things, we
have done so by identifying different aspects of control and by considering key
ways of investigating who exactly has control over something. Let us now turn
to the central question of why control is regarded as being so important. Why
care about it? What is its ethical importance?
5.61 In discussing this, we can return to the distinction between instrumental and
non‐instrumental value that was central in the previous chapter. We can ask if the
value of control and its different dimensions is primarily instrumental in nature.
That is, do we primarily value control because it is a means to other ends? Or
does control sometimes also have some form of non‐instrumental value or
ethical importance? Does control perhaps have both instrumental and non‐
instrumental value?
5.62 We can also ask a more general philosophical question here. Namely: does
it matter whether the value of control is primarily instrumental or whether it also
has some non‐instrumental significance? If so, why? We can here revisit the
classic dialogue The Republic, which was briefly discussed in Chapter 2. In that
book, Plato suggests that the things that have the greatest value and are most
important in life are those that have both instrumental and non‐instrumental
value. The key example Plato discusses is the virtue of justice.
5.63 In the dialogue, in which Socrates is one of the main characters, Plato lets
Socrates argue that the virtue of justice has both instrumental and non‐
instrumental value. In a similar way, it may be possible to argue that, depending
on who the subject of control is and what the object of control is, control can
have both instrumental and non‐instrumental value. (More on this below.) So, by
the criteria for being among the most important values in life that Plato discusses
in the Republic, this would mean that control of certain forms can be among the
most important values there are.
5.64 Regarding the value of control, it is also interesting to note that control
seems to be something that might be seen as being ambiguous in terms of its
value. That is, control can sometimes be regarded as having positive value but
sometimes also be regarded as having a more negative value. This, too, seems to
depend on who the subject of control is and what the object of control is. In
other words, an important critical question to always ask when it comes to
control is “who is controlling what or whom?”
5.65 One key example of when control seems to have a negative ethical value is
when somebody has control over another individual in the sense of having some
problematic form of power over that other person. As we already mentioned at
the end of the previous chapter, there is often something vicious or ethically
problematic about wanting or trying to control other human beings—at least if
those are mature, adult human beings who should be in control over their own
lives. So, it would seem that the more it happens that behavior change
technologies and other technologies are used to gain control over the behavior of
other people, the more ethically questionable these technologies become. And
the greater the potential is that this control will have a negative ethical value.
5.66 In contrast, when it comes to having control over oneself, or over certain
behavioral patterns of ours, this can have a very positive value. There is a long
tradition of placing a high value on self‐control—which has long been regarded
as a key virtue. Control over oneself and one's behaviors can be thought to both
have instrumental and non‐instrumental positive value. Clearly, it can have
instrumental value if the behaviors we are able to get greater control over help us
to achieve important goals, such as health or happiness. But a case can also be
made for the claim that control over oneself and one's own behavior patterns can
also have a non‐instrumental value.
5.67 Why might control (at least control over oneself and one's own life) be
thought to have a non‐instrumental value? One possible answer is that control
over oneself—self‐control—can be seen as a virtue. Virtues are sometimes seen
as valuable in a non‐instrumental way. The ancient Stoics, for example, thought
of self‐control in this way, as we saw in Chapter 2.
5.68 Another answer is that we might think of control over oneself, one's life,
and one's behavior as being a key part of personal autonomy. We typically think
of personal autonomy as having a non‐instrumental value of an ethically relevant
kind. By personal autonomy philosophers and other ethics researchers usually
understand being able to live on one's own terms, according to one's own goals,
principles, and values. Having control over oneself and one's patterns of
behavior is not the only thing involved in personal autonomy in that sense. But it
is one key aspect of personal autonomy. So, if we regard personal autonomy as
having a non‐instrumental value, and we regard control as one key component of
personal autonomy, then this seems to entitle us to attribute a certain amount of
non‐instrumental importance to control.
5.69 Control over oneself—as well as over instruments or other technologies—
might also be seen as having a non‐instrumental value for another reason and in
another sense. Namely, it might be seen as something that is part of mastery or
virtuosity. These are also things that people tend to admire and value as goals or
ends in themselves.
5.70 Think, for example, of gymnasts and other great athletes, like Simone Biles.
Biles has won more Olympic Gold medals than any other gymnast. One of the
things that are assessed in gymnastics competitions, and that people admire in
great gymnasts like Biles, is the control that the athlete displays over their own
body and movements. Or consider a virtuoso violin player—such as Itzhak
Perlman or Anne‐Sophie Mutter—or a virtuoso jazz musician—such as the alto
saxophone legend Charlie Parker. One, but certainly not the only thing that
people admire about such great musicians—as well as about other virtuosos—is
the control that they have over their instruments. When we admire different
forms of control that people might have, we seem to prize control not only as a
means to other ends. We also typically admire it as valuable and impressive in
itself. We attribute to it a certain amount of non‐instrumental value.
5.71 When we philosophize more generally about control and its value, it is also
worth reflecting on a claim made by the American philosopher T.M. Scanlon. In
his 1998 book What We Owe to Each Other, Scanlon claims that to understand
the value of something is to know how to properly value it. He thinks, for
example, that when it comes to some values, the proper way to value them might
not necessarily be to try to maximize them, but perhaps rather to cherish or to try
to protect existing instances of them.
5.72 Consider friendship as an example. To properly understand the value of
friendship, it might be thought, is not to try to maximize the number of friends
one has, but instead to cultivate a smaller number of close friendships. Or, to
take a couple of examples the philosopher Seana Shiffrin uses in another
context: to value particular meals or interesting conversations might not need to
involve wanting them to go on forever. This might be different than what is
involved in valuing certain particularly significant works of art. Some painting—
e.g., the Mona Lisa—might be an instance of something we think it is proper to
preserve for future generations.
5.73 What about control? What is the proper way to value it? Should we
maximize it, cherish it, cultivate it, conserve it, enjoy it while it lasts, or what?
This is a hard question. But it is worth noting that there is a widely shared
common‐sense idea according to which it is important to exercise some degree
of temperance with respect to how we value control. There is the familiar idea of
the “control freak,” that is to say, somebody who appears to value control too
much. According to this way of thinking, there is nothing wrong with valuing
control, even to some significant extent. But there are limits to how much
control we should want to have in order to not be control freaks.
5.74 One way of connecting this to some more traditional philosophical ideas is
to relate this idea of the control freak to the so‐called unity of the virtues thesis
from ancient philosophy that was mentioned in Chapter 2. The unity of the
virtues thesis is the idea that different virtues, skills, and excellences need to
form a sort of unified overall set of mutually supporting and regulating aspects
of a good person. Too much bravery, for example, without a sufficient degree of
justice or kindness will be a vice rather than a virtue. In the same way, too much
focus on being in control, without sufficient concern with other goals and values
as well, might make one into a control freak rather than somebody who is
concerned with being in control in a reasonable and virtuous way.
5.75 This is not the place to enter into an even more detailed discussion of how
exactly one should value control and what the right degree of concern with
control is. But it seemed worth mentioning in this context that if we take control
to have both instrumental and non‐instrumental value—as it is plausible to do—
then this does not necessarily mean that we need to think that control is
something to be maximized nor something that should always be given the
highest possible priority. Rather, control might reasonably be thought of as
something that should be valued within certain limits, in a way that also takes
other values into account, and that does not go overboard so that we start looking
like control freaks.
5.7 Concluding This Chapter
5.76 Let us return to the example we started out with. We considered a
smartwatch that is supposed to help the user to become a better version of him or
herself. If such a technology could help the user to, for example, exercise more
regularly and effectively, that smartwatch is an instance of the sort of pattern‐
changing technology that we have been understanding behavior change
technologies as being in this chapter. And if changing some of the patterns in the
user's behavior might lead to outcomes such as better health, the increased
control over those patterns of behavior might indeed have some value for the
user.
5.77 But there might also be a threat related to control in the case of something
like a smartwatch. Other people—e.g., the company behind the smartwatch or
their business associates—might start becoming agents of certain kinds of
control over the user. They might, for example, become capable of monitoring
and influencing many aspects of the user's day‐to‐day activities without the user
being aware of it. They might become capable of influencing the user to behave
according to their preferred patterns of behavior, rather than the user's own
preferred patterns of behavior. So, as the Dutch philosopher of technology
Marjolein Lanzing argues in her work on self‐tracking technologies, there might
be a significant threat to the user's personal autonomy here.
5.78 In general, unless we create the behavior change technologies that we use
ourselves, most behavior change technologies will be likely to have this dual
relationship to control and personal autonomy. This becomes clear if we keep in
mind that the different dimensions of control can be distributed across different
people. Behavior change technologies will sometimes enable users to do better
on some dimensions of control. But they will then typically also enable others
(e.g., the tech companies offering the service or others acquiring data from them)
to monitor users, influence them, and in other ways achieve some of the different
aspects of control discussed earlier.
5.79 How much control we should wish to retain in our own hands is, as noted
toward the end of the foregoing section, a difficult question to make general
claims about. Having control over our own behavior patterns is something that
can be regarded as both instrumentally and non‐instrumentally valuable from an
ethical point of view, because of how this is one aspect of personal autonomy. It
can also be valued as being part of the virtue of self‐control. Moreover, having
control over ourselves or over some instrument or some technology can also be
admired as a form of mastery or virtuosity, which can be valued in its own right
or for its own sake. But as we also saw earlier, there is also the issue of finding
the sweet spot where we value control in the right way and to the right degree,
and thereby avoid becoming control freaks who care too much about controlling
everything within our lives.
5.80 There are also important ethical questions to be asked about specific kinds
of behavior change technologies or methods—such as ones involving
gamification. If we change people's motivation for engaging in certain activities
—for instance, by stimulating a sense of competitiveness rather than an intrinsic
interest in something—when is that acceptable and when might that be ethically
speaking problematic? These are all fascinating issues that deserve a much
longer discussion than we were able to have in this chapter. Rather than
discussing them further, however, we will now instead turn to yet another part of
technology ethics in which control plays a crucial role. We will consider the
issue of responsibility. In particular, we will consider the worry that many people
have that some technologies might give rise to troublesome gaps in
responsibility.
Annotated Bibliography
Dorothea Debus (2016). “Shaping Our Mental Lives: On the Possibility of
Mental Self‐Regulation,” Proceedings of the Aristotelian Society, vol. CXVI,
no. 3, pp. 341–365.
A fascinating discussion of how we can have indirect control over aspects of
our own mental lives that we have no direct control over, such as our moods.
Ezio Di Nucci (2020). The Control Paradox, London: Rowman & Littlefield
International.
A wide‐ranging discussion about when and whether we can increase the
overall amount of control that we have by giving up direct control of certain
things.
Fleur Jongepier and Michael Klenk (editors) (2022). The Philosophy of Online
Manipulation, London: Routledge.
A collection of essays on the ethics of different forms of technological
manipulation.
Marjolein Lanzing (2019). The Transparent Self: A Normative Investigation of
Changing Selves and Relationships in the Age of the Quantified Self, PhD
Thesis, Eindhoven University of Technology:
https://pure.tue.nl/ws/portalfiles/portal/140533915/20191211_Lanzing.pdf.
An excellent discussion of the ethics of self‐tracking technologies.
Philip Pettit (2012). On the People's Terms: A Republican Theory and Model of
Democracy, Cambridge: Cambridge University Press.
Presents, among other things, an account of what it is for people to be in
control of the laws in the state they live in.
Plato (2007). The Republic, London: Penguin.
One of the greatest classics in the history of philosophy, which discusses,
among many other things, what kinds of values are the most important types
of values in life. Argues that both persons and states can be just and that in
both cases, it is both instrumentally and non‐instrumentally good for the
person or state to be just. Understands the virtue of justice as a certain form
of self‐constitution and as harmony among different parts of a person or a
state.
Filippo Santoni de Sio and Jeroen van den Hoven (2018). “Meaningful Human
Control over Autonomous Systems: A Philosophical Account,” Frontiers in
Robotics and AI, vol. 5, p. 15, doi: 10.3389/frobt.2018.00015.
Proposes a “track‐and‐trace” theory of meaningful human control over
autonomous technologies.
Thomas Michael Scanlon (1998). What We Owe to Each Other, Cambridge,
MA: Harvard University Press.
A wide‐ranging discussion of different aspects of ethics, including what value
is and how to properly value something.
Daniel Susser, Beate Roessler, and Helen Nissenbaum (2019). “Technology,
Autonomy, and Manipulation” Internet Policy Review, vol. 8, no. 2, pp. 1–22.
Offers an account of the ways in which online manipulation can pose a threat
to personal autonomy.
Notes
1 You can watch this commercial here: “Apple Watch Series 4—A Better You—
Apple—Adfilms, TV Commercials, TV Advertisements”:
https://www.youtube.com/watch?v=0cBJBj_tbHM
2 The idea of self‐tracking is sometimes associated with the notion of the
“quantified self”: a self‐image based on data collected through various kinds
of self‐tracking technologies. Here is a humorous depiction of the quantified
self, by students at the Dutch Hanze University of Applied Sciences in
Groningen: “Challenge Yourself Quantified Self”
https://www.youtube.com/watch?v=DeeCvmPN_cY. For more about what
those who take this general idea seriously think is valuable about it, see the
website of the “Quantified Self Institute”: http://qsinstitute.com
3 On this topic, see Carissa Véliz's TED talk in this video presentation: “The
Case for Ending Data Economy”:
https://www.ted.com/talks/carissa_veliz_the_
case_for_ending_data_economy
4 For more information about the idea of persuasive technologies, see this
webpage from the Center of Humane Technology:
https://www.humanetech.com/youth/persuasive‐technology
5 For more on the idea of gamification, see the “what is gamification?” section
of this website (http://gamify.com), which is very enthusiastic about
gamification: https://www.gamify.com/what‐is‐gamification?hsLang=en‐au
6 Frederick E. Allen (2011) “Disneyland Uses ‘Electronic Whip' on Employees,”
Forbes: https://www.forbes.com/sites/frederickallen/2011/10/21/disneyland‐
uses‐ electronic‐whip‐on‐employees/?sh=6d42cd3051b3
6
RESPONSIBILITY AND TECHNOLOGY: MIND
THE GAP(S)?
6.1 Two Events
6.1 In March 2016, an artificially intelligent computer program—“AlphaGo”—
faced off against the world champion of the ancient game of Go. In preparation,
the engineers behind the computer program had trained AlphaGo in two ways:
on a huge data set of existing Go games and by letting AlphaGo play millions of
games against itself. This way, the computer program was able to develop
strategies for how to play Go well and thereby get ready for the challenge. The
world champion of Go—or, perhaps one should say, the human world champion
of Go—was a Korean man named Lee Sedol. Sedol and AlphaGo played a total
of five games. Sedol won one of those games. But the other four games were
won by AlphaGo. So, just like the computer program Deep Blue had beaten the
human world champion of chess—Garry Kasparov—in 1996, now the world
champion of Go had also been beaten by an artificially intelligent computer
program.1
6.2 One interesting thing was that this was a computer program and not a robot.
And so a human being had to perform the part of moving the stones around on
the Go playing board. That human being—one of the DeepMind employees
working on the project—did not understand the strategies of the AlphaGo
program. Instead, he simply followed the instructions about what to do that the
program gave to him. He himself could never have won a single game of Go
against Lee Sedol. His only contribution was to move the pieces around on the
board. So this person, it might be thought, does not deserve credit for this
victory. He could just as easily have been replaced with robotic arms that would
move the stones around.
6.3 What about the people who built and designed the computer program? Did
they deserve the credit for the victory? This might also not be altogether clear.
They had not played the games of Go that were put into the training data. And
they did not play those millions of games that AlphaGo played against itself in
order to develop its strategies. Nor did they understand the strategies that the
computer program developed and then used. So, while they might deserve credit
for having developed a computer program with these capabilities, they might not
deserve credit for the victory in the Go games. Perhaps this was a game with a
loser—Lee Sedol—but with no winner. There may have been no one or nothing
that could be said to be praiseworthy for this Go victory.
6.4 Consider next an event that took place almost exactly two years later, i.e., in
March 2018. As was noted in Chapter 3, it then for the first time happened that a
pedestrian was hit and killed by a self‐driving car. The ride sharing company
Uber was test‐driving an experimental self‐driving car on a public street in
Tempe, Arizona. A “safety driver” was in the car: a human being who was in
charge of monitoring the performance of the car and who was supposed to stop
the car in case anything dangerous or risky was happening. However, when a
woman named Elaine Herzberg, who was walking with a bicycle, suddenly
crossed the street in front of the self‐driving Uber car, neither the artificial
intelligence in the car nor the human safety driver reacted quickly enough. The
image recognition software in the car switched back and forth between
classifying Herzberg as an unknown object, a bike, and a vehicle, and no
appropriate action was taken in time. The human safety driver noticed Herzberg
too late, and could only watch in horror as the car crashed into Herzberg.
Herzberg died in the ambulance on the way to the hospital.
6.5 Who was responsible for this tragic accident? The safety driver? The people
who designed the self‐driving car? The Arizona politicians who permitted the
testing of experimental self‐driving cars on public roads? Or perhaps the
company Uber, which was experimenting with such cars on public roads?
Different views are possible here. One view is that there is a gap in
responsibility. That is, one possible view is that this is a case where there is a
victim, but where it is not clear who the blameworthy party is. Just like it might
be unclear who deserved credit (if anybody!) for the victory in the game of Go
when Lee Sedol was defeated by AlphaGo, it might also be unclear in some
cases—e.g., the case with Elaine Herzberg—who deserves blame when
somebody is harmed by an autonomously operating technology, such as a self‐
driving car.2
6.6 This chapter discusses responsibility in the context of technologies. It has a
special focus on different kinds of worries that we might have about different
forms of gaps in responsibility that there might seem to be in cases where
artificially intelligent technologies produce good or bad outcomes. The chapter
will consider different types of responsibility—both for good outcomes and bad
outcomes. And it will discuss different philosophical views about how
technology might create confusion in relation to our normal views about
responsibility.
6.7 This is a topic that is almost always discussed within technology ethics. The
question “who is responsible?” is one of the first questions that always come up
when something is happening in a new and perhaps unexpected way, which can
happen when some new technology is introduced or when some new social
practice is introduced. So a book about technology ethics—especially an
introduction to technology ethics—cannot avoid discussing this topic.
6.8 Some people who write about these issues think that worries about potential
gaps in responsibility are over‐blown. Some even think that we worry too much
about finding people to hold responsible for things that happen, so that it would
be good if technologies created gaps in responsibility. Others think that the
practice of holding people responsible for good or bad outcomes is an essential
human practice, which is at the very heart of ethics, and that any disruptions of
these responsibility practices caused by technologies are very serious.
6.9 The purpose of this chapter is not to defend any particular view of the matter,
but rather to consider a number of perspectives on this issue. The chapter will
betray some inclination on the part of the author of the book to think about
responsibility for technologies in a certain way—namely, on the model of
human‐machine teams, where the humans are viewed as a form of supervisors or
managers who can be held responsible. But other views will also be given equal
billing and be taken equally seriously. As with other chapters in the book, the
aim of this one is also to stimulate the reader to think about this issue and to try
and make up their own mind about it.
6.10 One message in this chapter is that when it comes to responsibility and
technology, we should not only think about bad outcomes and blame and
punishment (i.e., in terms of a negative form of responsibility). We should also
consider positive forms of responsibility. For instance, who deserves credit when
something good happens? Both forms of responsibility are important to us
human beings, though in different ways. We typically want to be praised when
something good happens—we typically want to take credit for good things,
when it is within the realm of the conceivable that we could be given some
credit. But on the other hand, we typically do not like being blamed for bad
things that happen. We usually then try to pass on blame to others or find
excuses for why we should not be blamed. When we think about technology and
responsibility, these asymmetric attitudes that many of us have in relation to
praise and blame are important to keep in mind.
6.2 What Is Responsibility? Different Ways in
Which People Can Be Held Responsible and
Different Things for Which People Can Be Held
Responsible
6.11 Philosophers have written a lot about issues having to do with
responsibility. They have drawn many distinctions worth keeping in mind, and
twisted and turned the issue of responsibility in all different directions
imaginable. So, there is a lot of prior work on this topic that those who are
interested in technology and responsibility can make use of. In fact, there is so
much prior work on issues related to responsibility that it can be hard to know
where to start. There are many different interesting questions that one can
discuss related to responsibility. For example, what is the relation between our
ideas about responsibility and our beliefs about free will and determinism? Here
we will focus on some of the distinctions and ideas about responsibility that are
most commonly discussed in technology ethics.3
6.12 One basic distinction to keep in mind is the distinction between moral
responsibility, on the one hand, and legal responsibility, on the other. Legal
responsibility is defined in relation to the existing laws in a particular society.
Laws may differ among different countries. And laws may change over time. So,
what one is legally responsible for in one society at one point in time might be
different from what one is legally responsible for in another society at a different
point in time, or what one is responsible for within the same society at different
points in time.
6.13 We will mostly set issues related to legal responsibility aside and instead
focus on moral responsibility: what one can be held responsible for from an
ethical point of view. This may or may not match up with what one is legally
responsible for within a particular society at a particular point in time. A focus
on moral responsibility makes sense when we discuss technology ethics since
technological developments can sometimes be quicker than legal developments.
There can be cases, in other words, where there is not yet any clear legal basis
for holding people responsible for something that they do, but it may already be
possible to discuss whether they bear a moral responsibility for what they are
doing or have done.
6.14 For example, during the late 2010s and early 2020s, social media
companies were increasingly being put under public scrutiny for the effects that
their social media websites were having on society as a whole. But the legal
situation was not always clear—there was a lack of regulation—yet this did not
preclude the possibility that the social media companies bore a moral
responsibility for the outcomes that their social media helped to bring about.
6.15 We are focusing on moral responsibility here, then. Moral responsibility,
some philosophers argue, is a phenomenon with many different “faces.” David
Shoemaker, for example, argues that we should distinguish among the
“attributability,” “answerability,” and “accountability” faces of responsibility.
We can ask to whom we should attribute a certain outcome: who is behind a
certain outcome, so to speak? And to what aspect of the person can we attribute
an outcome (their underlying character or personality, or to an uncharacteristic
temporary impulse or perhaps even momentary insanity on their part)? Who can
answer questions about something that has happened? And who can be held to
account for what has happened? The answers to these questions may all point in
one and the same direction. But then again, they may not.
6.16 At any rate, according to Shoemaker, when we think about responsibility,
we should keep these different “faces” of responsibility apart. As we will see
later, some technology ethics researchers think that this is a very important
insight. Daniel Tigard, for example, who was a student of Shoemaker's, argues
that when we think about AI, robots, and responsibility, we might be able to hold
these technologies responsible for what they do, at least from the point of view
of some of the so‐called faces or different aspects of responsibility.4
6.17 Here are two other distinctions that are often discussed within technology
ethics and that are important to have clearly in mind. One we have already
alluded to above. It is the distinction between responsibility in a more positive
sense and responsibility in a more negative sense. When good things happen,
this raises positive questions about responsibility: who is behind the good
outcome, and who can we praise or reward for this, and so on? When bad things
happen, in contrast, this raises what might be called negative questions: who is
behind the bad outcomes, who can answer questions about what went wrong,
and who can be blamed or punished for what has happened?
6.18 Most philosophical discussions about responsibility—both in ethics more
generally and within technology ethics more specifically—tend to be about
negative responsibility (bad outcomes, blame, punishment, and so on). But it is
important to keep in mind that responsibility also has a more positive side to it.
We also make good things happen. And we certainly tend to want to take credit
for good things that happen and might be regarded as being our doing.
6.19 The other distinction that technology ethics researchers frequently discuss
is the one between what is often called backward‐looking and forward‐looking
types of responsibility. After something has happened (something good or
something bad) one can ask who was responsible for what happened. This is
responsibility in the backward‐looking sense. But when we look into the future,
and we think about different things that could happen and different ways in
which things could go, we might also think that there are people whose
responsibility it is to make sure that certain good outcomes are brought about
and that certain bad outcomes are avoided.
6.20 For example, when we think about climate change and the world we are
leaving behind for future generations, one of the questions that sometimes comes
up is whose responsibility it is to try to make sure that we do not wreck the
climate for the people of the future. Is it the responsibility of politicians in
power? Is it each person's individual responsibility? Is it a collective
responsibility that we all share? Or what? That is one example of a question that
relates to so‐called forward‐looking responsibility.
6.21 We can also put these questions and distinctions together in different ways.
For example, we can ask backward‐looking questions about both negative and
positive responsibility: e.g., who is to blame for something bad that has
happened or who is to praise for something good that has happened? Likewise,
we can ask both forward‐looking questions about negative and positive
responsibility: e.g., who is responsible for making sure that negative outcomes
are avoided and who is responsible for making sure that good outcomes are
achieved?
6.22 These four different kinds of responsibility questions might potentially
correspond to different kinds of gaps in responsibility that one might worry
about in the context of technology ethics. We will look at the idea of potential
gaps in responsibility—or “responsibility gaps,” for short—in greater detail
below. But for now, the general idea is this: when we start using technologies
that take over tasks from human beings, like robots and other AI technologies
do, there are often worries that there might be cases when the stakes are high and
when outcomes might come about for which somebody should intuitively
speaking be held responsible. But it might be unclear who, if anybody, could or
should be held responsible. So potential responsibility gaps occur.
6.23 Such gaps could relate to both backward‐looking and forward‐looking
responsibilities. And they could relate to both negative and positive
responsibilities. Hence, we can imagine at least four types of potential gaps,
which will be explained in the next section. These four types of responsibility
gaps are shown in the table below:
Responsibilit Backward‐looking
y gaps
Negative
Backward‐looking negative
responsibility gaps
Positive
Backward‐looking positive
responsibility gaps
Forward‐looking
Forward‐looking negative
responsibility gaps
Forward‐looking positive
responsibility gaps
6.3 Responsibility Gaps: General Background
6.24 Why might it be thought that technologies could create gaps in
responsibility? To understand this, it is a good idea to first reflect on general
ideas that many people have about what makes people responsible for things.
Again, we will consider both positive responsibility and negative responsibility.
That is, why do we sometimes think that people deserve praise for their
achievements? And why do we sometimes think that people deserve blame or
other forms of criticisms for bad things they do or bad outcomes that have come
about?
6.25 Of course, as with many things, there are different theories about these
matters. We will not be able to consider all of them here. Instead, we will focus
on theories that seem especially relevant in this context. Let us first consider
positive responsibility in the backward‐looking sense. In other words, there is
something good that has happened, and somebody deserves credit for it.
6.26 One useful theory that one can look at here is the philosopher Gwen
Bradford's theory of what makes something count as an achievement. Another
useful theory is Hannah Maslen and her colleagues' theory of praiseworthiness.
It seems appropriate to marry these two accounts together since achievements
are, in essence, a positive manifestation of responsibility. Putting together these
theories, we get something along the following lines.
6.27 Whenever we are assessing the value of somebody's achievements, the
following four variables make sense to keep in mind:
The value of the output produced: the better some output is, the more it makes
sense to think of it as being an achievement worthy of praise.
The nature of the contribution of the person(s) under consideration: the more
significantly somebody has contributed to the production of some output—and
the less it is a matter of luck that the output was produced—the more it makes
sense to view it as an achievement worthy of praise.
The cost of the commitment on behalf of the person(s): typically, the more effort,
time, attention, and stress had to be exerted, the more it makes sense to view
something as an achievement worthy of praise.
The voluntariness of the commitment and efforts of the person(s): the more it is
the case that one has voluntarily determined to pursue some outcome, the more it
makes sense to view it is an achievement worthy of praise.
In short, if there is a good output, and a person or group of persons has
contributed significantly to its production, and done so in a way that involved
commitment/effort and voluntary determination, then it usually makes a lot of
sense to view the output as an achievement. It makes sense to view it as an
achievement on the part of the person or group of persons for which they can be
praiseworthy and deserve credit.
6.28 On the flipside, when one or more of these conditions do not hold, it
becomes less clear whether something is a praiseworthy achievement for which
somebody can be seen as being positively responsible in the backward‐looking
sense. Imagine, for example, that something good has happened (e.g., a patient
has recovered from an illness) and that somebody tries to claim credit for this
(e.g., the doctor in charge of treating the patient), but that several of the above‐
mentioned conditions do not obtain. Perhaps the actions of the person wanting
credit (the doctor) did not really contribute much to the realization of the
outcome. Nor did the person put in much of an effort (e.g., the doctor put in a
minimal effort), and nor did they put in that minimal effort very gladly or
voluntarily (e.g., the doctor only took any kind of action when others directly
asked him or her to do so). In such a case, credit does not seem to be due to the
person, and the good outcome (the patient's recovery) was a matter of luck: a
nice outcome that perhaps nobody deserves praise or credit for.
6.29 Consider next negative responsibility in the backward‐looking sense:
somebody deserving to be blamed or punished for something bad that has taken
place. Here, too, there are different theories about what conditions should obtain
in order for it to make sense to single out somebody as being deserving of blame
or even punishment. We will not be able to consider all of them. But let us focus
on the following common ideas about this.
6.30 It is often thought that if the following conditions obtain, then somebody
can be seen as being deserving of blame or punishment:
There is something bad that has happened or that somebody has done. There
were other things that could have happened instead.
There is somebody who faced a choice and who had some control over what
outcome would occur. That person did not exercise their control over the
situation so as to avoid the bad outcome.
The person was able to understand and predict that this outcome was possible,
and they were able to assess how likely it was to occur unless they took some
sort of action.
The person was able and was in a position to make a moral judgment about
whether it would be a bad or negative thing if the bad outcome were to come
about.
In the abstract, these kinds of conditions can sound very, well, abstract. But as
soon as we consider concrete cases, these kinds of conditions for backward‐
looking negative responsibility start making clear sense. Suppose, for example,
that we are thinking about a doctor and a patient again. This time, the outcome
was a bad one: the patient ended up dying.
6.31 We can consider the real‐life case of when perhaps the most famous
singer/entertainer from the 1980s to the 1990s, Michael Jackson, died of a drug
overdose of propofol in 2009. Many people blamed Jackson's doctor, Conrad
Murray. Dr. Murray had some control over what could happen, because he was
supplying Jackson with the drug that killed him, and he could have refused to do
so. As a trained physician, Murray was able to understand and should have been
able to predict that the setting in which he was giving this powerful drug to
Jackson was a dangerous setting: it was in Jackson’s home, and not in a hospital
operating room, which is where this drug is usually used. Lastly, Murray was, it
might be thought, in a position to appreciate that it was a bad and dangerous idea
to administer this powerful drug to Jackson in this setting. Accordingly, when
Jackson died after having taken this drug in his home, many people blamed Dr.
Murray. He was punished with a prison sentence, and was also morally blamed
by many of those who learned about what had happened, at least as it was
described during the ensuing trial.5
6.32 In contrast, in cases when bad things happen, but one or more of the three
other conditions do not obtain, then it might feel like there ought to be somebody
who should be held responsible for what happened. But there might not be
anybody who can fittingly be held responsible. There is, instead, a gap. To use
an example similar to the one we just looked at: when Michael Jackson's great
musical “rival,” Prince, died in 2016 while he was alone in his home, also of an
accidental drug overdose, many people felt that there must be somebody who
can be blamed or punished for this. Prince's cousin, Charles “Chazz” Smith, for
example, started the project “justice4cuz” to try to get clear on whose
responsibility it was that his world‐famous cousin, the music icon Prince, had
died of an accidental fentanyl overdose.6
6.33 Perhaps this was somebody's fault. But there was an investigation by the
Carver County police department. And they were not able to find anybody they
could point a finger at. In the end the death investigation was ended.7 Perhaps
this was nobody's fault, but a tragic accident. Perhaps there was a gap in
responsibility. We will now soon get to how technologies—in particular,
artificially intelligent or autonomous technologies—might create such gaps. But
let us first briefly consider forward‐looking positive and forward‐looking
negative responsibilities.
6.34 If somebody is seen as having a responsibility to help to make sure that
some good outcome comes about, which has not yet happened, it is often
because that person is seen as standing in some special relationship with one or
more other persons, so that they have special duties to promote the well‐being of
the other. For example, parents are seen as having a duty and a responsibility to
make sure that things go well for their children. Teachers might be seen as
having responsibilities, to some extent, for enabling the success of their students.
And doctors, to use that example again, are seen as having a responsibility to “do
good” to their patients.
6.35 When it comes to responsibilities to make sure that harms or bad outcomes
are avoided, it is less common to view these as typically being tied to personal or
other special relationships. Rather, we are usually seen as having a general
responsibility to try to avoid creating situations in which other people are
harmed or in which things go badly for others in other ways. This is the flipside
of backward‐looking negative responsibility: if it later turns out that we are
blameworthy for something bad that happened, where we could have taken steps
to avoid the risk of harm, we realized the risk, and we realized the severity of the
situation—well, then it is also the case beforehand that we now have a duty or
forward‐looking responsibility to try to avoid having these bad things happen.
6.36 With respect to forward‐looking positive responsibilities, we might have
gaps if there are good outcomes that it would be nice if somebody were to bring
about but no special relationships to which we can relate special responsibilities
to promote those good outcomes. With respect to forward‐looking negative
responsibilities, we might have gaps if there is not what philosophers call a
generally responsible “moral agent” there, i.e., a person of the sort that can have
duties and that can fittingly be held responsible if they fail to do their duty. Let
us now relate these general ideas to cases in which technologies might be
thought to create gaps in responsibility.
6.4 Responsibility Gaps Created by
Technologies
6.37 One domain of life where there might be reason to be worried that
backward‐looking positive responsibility gaps—what can also be called
achievement gaps—might occur is in the domain of work. Artificial intelligence
and other advanced technologies are increasingly being introduced into different
types of work contexts, where positive responsibilities were previously
completely associated with human efforts or human ingenuity.
6.38 Think, for example, of decision‐making tools that can help medical doctors
to diagnose illnesses, e.g., by looking at X‐ray images of patients and suspected
problems they have. AI tools might come to outperform human medical doctors
both in terms of diagnosing the problem and in coming up with a treatment
regime. The doctor's role might be reduced to communicating the findings of the
AI system to the patient. Or it might even be reduced further than that, because
perhaps the AI system can also come up with personalized ways of
communicating medical information to patients that are a perfect fit with the
individual patients' ways of receiving information.
6.39 Or consider legal work—e.g., looking up old cases, relating them to new
legal cases, and coming up with suggestions for how the old cases might serve as
precedents for how to judge the new cases. This might also be something that AI
systems might at some point become better at than human beings. Or consider
the manufacture of any kind of product whose production we now commonly
associate with the exercise of human skill and mastery. Here, too, we can
imagine that machines might become better than us humans at producing things
that people might be interested in having or buying.
6.40 All these kinds of cases or others similar to them might reduce the room
there is for human achievement in the workplace. Human beings might not have
more to do than to maintain or supervise or perhaps fix the
technologies/machines/AI‐systems that do the difficult and impressive parts of
the work, which was previously done by highly skilled human beings. Where
there was previously room for human achievement—for being considered
responsible for the outcomes of work in a positive sense—there may come to be
significant gaps in responsibility. This might undermine the meaningfulness of
the work we do. After all, meaningful work is often associated with work that
involves contributing significantly to bringing about good outcomes (e.g.,
diagnosing illnesses and recommending treatments, engaging in creative legal
reasoning, or producing important goods).
6.41 Technologies in the workplace, then, might create very good outcomes, and
they may outperform human beings who previously performed these tasks. But
in the process, they might create achievement gaps: they might make it
impossible, or at least very hard, for human beings to be able to claim positive
credit for good outcomes produced in workplaces.
6.42 The example with AlphaGo from the introduction to this chapter might help
to further illustrate this idea. In that case, as we saw above, the computer
program was trained on past games and by playing games against itself. The
only role left for a human being was to move the playing stones around on the
board, and the best human player was defeated by the AI system. The human
being who moved the stones around was involved in these “victories” by
AlphaGo over Lee Sedol. But the person in question is unlikely to feel that it
was a great achievement on his part. And others are unlikely to give him credit
for the win. The same sort of thing could easily happen in many workplaces,
leaving us with achievement gaps and fewer opportunities, as a result, for doing
meaningful work for which we can justifiably feel positively responsible.
6.43 Before we get to backward‐looking negative responsibility gaps: how likely
is it that most jobs will be automated or mostly taken over by AI‐systems or
other technologies? A 2021 World Economic Forum report of expert opinions
co‐authored by the influential computer scientist Stuart Russell and the
economist Daniel Susskind states that “many AI experts predict that machines
may outperform humans at every task within 45 years.”8 That might be
exaggerated. And it should be noted that many AI experts are of a very different
opinion. However, this does give an indication of the fact that many experts from
different fields related to AI do take the prospect that machines will outperform
humans in a wide range of tasks very seriously. So, we should be asking
ourselves what this means for our own opportunities for human achievement and
excellence at our workplaces and elsewhere.
6.44 Let us now get to backward‐looking negative responsibility gaps. Even if
machines might become able to outperform humans at a wide range of tasks, and
even though machines may also already outperform us humans at many tasks,
this does not mean that these machines will always successfully carry out their
tasks in a way that is fully safe for human beings. Humans have already been
harmed by advanced technologies—e.g., self‐driving cars hitting and killing
human beings. And this will continue to happen. Some technologies—e.g., some
military technologies such as autonomous weapons systems—will even be
designed specifically to harm and even kill human beings. These kinds of
observations have led many to worry about what we are here calling backward‐
looking negative responsibility gaps. After something bad happens, such as
somebody being killed by a technology, it might seem like somebody should be
held responsible. But it might not be clear or even possible to answer who is
responsible.
6.45 For example, when the German philosophers Alexander Hevelke and Julian
Nida‐Rümelin discuss responsibility for harm caused by self‐driving cars in a
2015 article of theirs, they argue that the people using these cars cannot be held
responsible if the cars cause harm. After all, they are not driving and are not
exercising direct control over the car. Nor, according to Hevelke and Nida‐
Rümelin, can designers of such cars be held responsible for specific crashes that
their designs might cause, because it is not possible for them to predict what
exactly will happen once these cars are participating in real‐world traffic. The
focus of designers is, and should instead be, to try to create cars that are as safe
as possible (i.e., a forward‐looking positive responsibility). If something happens
that they could and should have predicted, then they might be held responsible.
But if and when situations occur that they did not and could not easily have
predicted, they are not responsible. Or so Hevelke and Nida‐Rümelin argue.
Others have made similar arguments.
6.46 One such argument comes from the Australian philosopher Robert Sparrow,
who is well known for raising skeptical worries about many AI and robot
technologies. He has argued forcefully that “killer robots” should not be
permitted because there are likely to be many situations in which autonomous
weapons systems will give rise to responsibility gaps. According to Sparrow,
designers, commanders, and others in key positions in relation to autonomous
weapon systems will not be able to fully predict or fully control what
autonomous military technologies do. And therefore, the key people who interact
with and use these technologies could not be fully responsible for any wrongful
harms that these technologies might cause.
6.47 The group behind the “campaign against killer robots,” which was
mentioned at the beginning of Chapter 2, presents similar arguments for why
they think autonomous military robots and other AI‐driven weapons systems
should be banned. It will not be possible, the campaign members argue, to have a
sufficient amount of “meaningful human control” over these systems. And it will
be unclear who is responsible when the technologies kill people. So, these
technologies should be banned, because it is important that it is clear who is
responsible after something bad happens, especially when it is not a pure
accident.
6.48 The general idea here—both when it comes to self‐driving cars and when it
comes to autonomous military robots—is that these technologies will function in
a way that is too far away from our immediate control, and they will function in
ways that we cannot predict well enough. Therefore, it will not be fair or right to
blame or punish any of us human beings when these technologies harm or kill
human beings. Or, at least, this would be so in a significant enough number of
cases that we would have to worry about these responsibility gaps being a type
of systemic problem associated with these technologies.
6.49 When it comes to cases in which we would want or find it appropriate to
punish somebody, but we could not find anybody who it would seem right to
punish, we would have what John Danaher calls a “retribution gap.” Punishment,
in particular, only seems justifiable in cases where people did know or could
control what was going on. The worry is that some autonomous technologies
will not always allow for those conditions to obtain.9
6.50 What, next, about forward‐looking positive and negative responsibility? Let
us start with the latter. If it could be the case that there would be a backward‐
looking negative responsibility gap after a technology such as a self‐driving car
or autonomous weapon system harms or kills somebody, then this might seem to
imply that there must also have been a forward‐looking negative responsibility
gap before this harming or killing happened. In other words, if somebody had
had a forward‐looking duty or responsibility to make sure that a technology did
not harm anybody, then surely that person could then be blamed after the fact if
the technology did indeed harm somebody. But assuming that there is a
backward‐looking responsibility gap here, then that might imply that before the
bad thing happened, there was potentially also a responsibility gap with respect
to whose responsibility it was to make sure that the bad outcome was avoided.
6.51 Consider next forward‐looking positive responsibility gaps that
technologies might create, or that might be associated with technologies. Such
gaps exist if the following conditions obtain: there are certain good outcomes
that could occur in the future, these outcomes are somehow closely related to
technologies, and it would seem that it would be right or good if somebody were
responsible for bringing about those good outcomes, but perhaps nobody could
actually be established as being responsible for doing this.
6.52 One reason such forward‐looking positive responsibility gaps could arise is
as follows. As noted above, responsibilities to try to benefit others are often
considered as being tied to special relationships, such as those between parents
and their children or doctors and their patients. If there are technologies that we
could develop/refine that could do a lot of good, but there are no such special
relationships that can be appealed to as reasons for why particular people have
responsibilities to try to do good for others, it might not be possible to find
anybody who can be said to be responsible for developing these technologies
that could do a lot of good. That might be so even if it might seem that it would
be good if somebody did have such a responsibility. This would be a forward‐
looking positive responsibility gap related to technologies we could develop or
further refine.
6.53 It might be thought, for example, that it could greatly benefit future
generations if certain technologies were developed, and that it would therefore
be a good thing if it were somebody's positive responsibility to create those
technologies. But if nobody stands in any special relationships to future
generations, perhaps nobody has a clear and strong forward‐looking
responsibility to develop these technologies that would benefit those future
generations. So in relation to these imagined technologies—whatever they might
be—there might be a forward‐looking positive responsibility gap.
6.5 Filling Responsibility Gaps by Having People
Voluntarily Take Responsibility
6.54 We have just considered potential responsibility gaps related to
technologies that might do good or that might cause problems. Let us now spend
the rest of the chapter reflecting on how one might try to fill responsibility gaps
and on whether, and if so why, it is important to try to fill these gaps. If, for
example, technologies like self‐driving cars, military robots, or other forms of
artificial intelligence become widespread, there might be many responsibility
gaps opening up of all the four general kinds highlighted above. And this might
seem problematic to a lot of people. So, it might seem like it would be good if
we could come up with different strategies for filling these gaps.
6.55 One possibility would be that people simply step forward and volunteer
themselves as candidates to fill responsibility gaps. People are sometimes
willing to take responsibility for something that has happened. Or people can be
willing to take responsibility for making sure that something will not happen or
for making sure that some good outcome will be achieved. Could this be a way
of solving if not all, then perhaps at least many responsibility gaps related to
technology?
6.56 One question here is how likely people are to be willing to volunteer
themselves as candidates to fill technological responsibility gaps. A closely
related question is what incentives they might have for or against doing so.
Another important question is whether people would deserve to be seen as
responsible for things that they might want to take responsibility for. Let us put
that last question aside for the moment and start with the former two. When we
think about those questions, two possible asymmetries in people's willingness to
take responsibility almost immediately come to mind.
6.57 The first asymmetry relates to the distinction between backward‐looking
positive responsibility and backward‐looking negative responsibility. The
asymmetry we can expect is this: people are more likely to be eager to take
responsibility for good outcomes caused by technologies than they are to take
responsibility for bad outcomes caused by technologies. If some AI system, for
example, helps to identify a medical problem and then also helps to suggest a
treatment regime, and the treatment ends up being successful, the doctor
involved in this procedure might be willing and indeed motivated to claim
responsibility or at least partial responsibility for these good results.
6.58 In contrast, if some military unit is using an autonomous military robot, and
that technology gets out of control and starts killing innocent civilians, then the
members of the military unit might be much less willing to and much less
motivated to take responsibility for this bad result. Or imagine that somebody is
riding in their self‐driving car, and the self‐driving car hits and kills somebody.
Then the person who was using the self‐driving car is unlikely to be as motivated
to take responsibility for this as they would be motivated to take responsibility
for some good outcome that some technology they might be using might bring
about.
6.59 Taking responsibility for good outcomes is more attractive than taking
responsibility for negative outcomes in many cases. The former involves
potentially being praised or rewarded for the achievement of the good outcome.
The latter involves potentially getting blamed or punished for the bad outcome.
So, it is no wonder that we should expect people to be much more motivated to
want to voluntarily fill backward‐looking positive responsibility gaps than
backward‐looking negative responsibility gaps.
6.60 A second asymmetry that we can expect is related to the difference between
backward‐looking positive responsibility gaps and forward‐looking positive
responsibility gaps. On the one hand, as just noted, many people are likely to be
willing to take responsibility for good outcomes that technologies have already
been brought about. But in contrast, on the other hand, most people may be
much less strongly motivated to take on the responsibility for taking active steps
to make sure that technologies will bring about positive outcomes. In other
words, while there might be no shortage of people willing to take responsibility
for good things that have already happened, there might be a shortage of people
who are equally willing to take responsibility for making sure that good things
will keep happening.
6.61 Why would one expect this second asymmetry? As noted above, if we take
responsibility for something good that has already happened, then if others are
willing to give us credit for the good outcome, they might be likely to praise us
or to reward us in various ways. And there might be no clear cost to us.
However, while people might praise us for taking on a forward‐looking positive
responsibility, we also then put ourselves under pressure to deliver that good
outcome. That might be costly in different ways. Among other things, it might
involve so‐called opportunity costs: we might be missing out on other things that
we could be doing instead. Moreover, we might also set ourselves up for
criticism in case we fail to deliver the good outcome. So, it might be “safer” to
let somebody else take responsibility for bringing about the good outcome
instead.
6.62 As we have just seen, only one of the four kinds of responsibility gaps
related to technologies is likely to be attractive to people who want to step
forward and claim or take responsibility: namely, backward‐looking positive
responsibility gaps. Is that a good way of filling such responsibility gaps?
6.63 Well, here it might be wondered whether the people who might step
forward and claim responsibility really deserve credit for the outcomes that
certain technologies (e.g., advanced AI systems whose inner workings are
opaque to us) have brought about. There is a difference between being willing to
take responsibility for some outcome, on the one hand, and deserving credit for
that outcome, on the other hand. In those cases where outcomes produced by
technologies are good and it is unclear who, if anybody, can take credit for the
good outcome, other people might not be willing to accept that people who wish
to take responsibility for those outcomes really deserve to be seen as responsible.
6.64 In short, the whole idea of filling responsibility gaps by having people
simply take responsibility for outcomes that technologies have caused or might
cause seems to be a problematic idea for all the reasons considered in this
section. Let us now explore other options.
6.6 Should We Perhaps Welcome Responsibility
Gaps?
6.65 Before we move on to exploring other options for how to fill responsibility
gaps, let us first consider another possible response to the whole idea of
responsibility gaps created by technologies. In particular we will consider a
suggestion made by John Danaher, who was mentioned above in relation to his
suggestion that technologies that cause harm might open up retribution gaps:
cases where we might think we should punish somebody, but in which it is
unclear who, if anybody, deserves to be punished. The idea that Danaher has that
we will consider in this section is this: perhaps it is a good thing that
technologies can create responsibility gaps and that there is sometimes no human
being that can rightly be considered as responsible for some outcome. After all,
being responsible for something can sometimes be burdensome. So perhaps we
should welcome the fact that technologies sometimes seem to open up gaps in
responsibility.
6.66 Danaher has two main arguments in favor of this view. The first one we just
implicitly considered, but let us make it explicit. Danaher argues that being held
responsible is often a burden, that it would sometimes be better if there were
fewer things we were being held responsible for, and that we can therefore
welcome some forms of responsibility gaps created by technologies. In
particular, being punished or blamed is highly burdensome and often unpleasant.
So, if we can avoid blame and punishment and we can say that a technology has
created a responsibility gap, Danaher suggests that this can sometimes be a good
thing.
6.67 The second argument Danaher offers is of a more “metaphysical” sort. It
relates to something that was briefly mentioned earlier in this chapter, namely,
the connection between moral responsibility and free will. Danaher is a skeptic
about free will, at least about free will in the sense associated with deserving
praise or blame for things that happen. According to Danaher, we can only
deserve, for example, blame or punishment if we bring something about out of
our own free will. But, according to Danaher, there are philosophical reasons for
being skeptical about the existence of free will. Therefore, there are also
philosophical reasons for being skeptical about people ever deserving to be held
responsible for what they do. So therefore, it is a good thing, Danaher argues, if
technologies create gaps in responsibility. It fits, according to this argument,
with the whole idea that since people lack free will in a strong sense, they are not
really responsible for what they do in the first place.
6.68 How should one respond to Danaher's suggestion that perhaps
responsibility gaps created by technologies are to be welcomed? And what
should we think about the arguments Danaher offers in favor of this suggestion?
Let us start with the second argument, i.e., the one having to do with Danaher's
skepticism about free will and the idea that responsibility requires having free
will in a strong sense. Why does Danaher think that people do not have free
will? His reasons have to do with the idea that free will seems to require
indeterminism (the idea that not everything is determined by natural laws) but
that he thinks that determinism is true (i.e., the idea that everything is determined
by causes and effects related to laws of nature). Whether we should follow
Danaher's lead and become free will‐skeptics based on philosophical ideas about
determinism is a bigger topic than we can quickly settle in this brief section.
However, here are some considerations to take into account.
6.69 Many philosophers reject the sort of incompatibilist view that Danaher
endorses (i.e., the view that free will and determinism are incompatible). They
instead adopt a so‐called compatibilist view (i.e., the view that free will and
determinism are compatible). So according to many other philosophers,
believing in cause and effect and the existence of natural laws and a
deterministic universe is somehow compatible with the existence of free will in
human beings. If they are right, we cannot escape responsibility for what we do
by claiming that the world is a deterministic system.
6.70 Another response is associated with the philosopher P.F. Strawson.
According to him, our everyday human practices of holding one another
responsible for things is actually quite independent of abstract philosophical
ideas about whether the world is deterministic or indeterministic and whatever
that might mean for the existence or non‐existence of free will in some strong
sense. The practice of holding each other responsible and sometimes accepting
or taking responsibility for things we do or things that happen are moral
practices that are part of being a social species, Strawson argues. It has to do
with the kinds of social interpersonal attitudes and reactions we have to each
other as human beings.
6.71 On this view, metaphysical views about causes and effects, natural laws,
and determinism are subjects for philosophy seminars, not for moral discussions
about who is responsible for what. So Strawson might say that we cannot settle
questions about responsibility and technology by bringing up abstract
metaphysical arguments related to determinism. Instead we need to settle these
questions using moral arguments about what is right and wrong in people's ways
of interacting with the technologies they invent or use.
6.72 Let us now return to the first argument from Danaher considered above for
why, according to him, we should welcome responsibility gaps opened up by
technologies: namely, the argument that being held responsible for something is
often a burden, and that it therefore can be a great relief if gaps in responsibility
are opened up by technologies. To this it might be responded that yes, being held
responsible in the negative sense is often a burden: it is not nice to be blamed or
punished.
6.73 However, being seen as responsible for something can also be nice in
different ways. Being praised or rewarded can be very nice in cases of positive
responsibility. We want people to recognize us as responsible for good things we
do or good outcomes that we are somehow associated with. So, if technologies
open up responsibility gaps, there is also something that we can lose that we
might find to be nice and positive: namely, opportunities for being credited with
good outcomes or good deeds, and the associated praise and the associated
potential rewards.
6.74 Moreover, even though it is not always nice for those who are blamed or
punished that they are blamed or punished, many people find it to be important
and valuable to be able to blame or punish people who appear to deserve to be
blamed or punished for what they have done. It is frustrating to many people
when it is not possible to identify any responsible agent whose fault it is that
some bad thing has happened. This does not necessarily require that that person
or those persons are punished in a retributive sort of way.
6.75 Let us, for example, return to the ubuntu ethics perspective that was brought
up in the chapter about what ethics is (i.e., Chapter 2). In ubuntu ethics, there is
the idea that people who are responsible for bad things can make amends and
apologize and then be welcomed back into the community. In other words, there
can be cases in which people have done bad things (e.g., by creating
technologies that create bad outcomes), where they are blamed for this, but
where they are not necessarily punished in a retributive sort of way but where
they instead make amends or try to repair the damage they have caused to their
relationships with other people through what they have done or been responsible
for.10
6.76 The perspective Danaher offers opens up big questions that we cannot
consider here in detail. So, let us set his view aside and focus on other
suggestions for how to try to fill responsibility gaps opened up by technologies.
Since this is an introduction to technology ethics, we are not trying to settle any
of the debates or questions brought up throughout the book. Rather, as noted
earlier, the aim of every chapter is instead to stimulate interest in these questions
in the reader and to open up discussion. Perhaps the reader agrees with Danaher,
perhaps the reader strongly disagrees with Danaher. Perhaps the reader disagrees
with all the different perspectives discussed in this chapter or in other chapters.
If so, the author of this book urges you to put pen to paper, or fingers to keys,
and to articulate your alternative perspective.
6.7 Responsible Machines?
6.77 Let us now consider other possible responses to the above‐described
worries about looming responsibility gaps. One such response is to turn our
attention away from human beings and instead consider whether technologies or
machines could themselves be responsible for what they do. Could we, for
example, ever have reason to praise a machine that produces some good
outcome or blame or punish a machine that produces some bad outcome? Could
it ever make sense to blame the robot, so to speak? In general, let us consider the
idea that machines might be responsible for what they are doing. This would be
a way of solving the responsibility gap problem, because the machine that might
seem to give rise to the problem would itself help to fill the gap.
6.78 Commenting on this idea, the German philosopher Christian List compares
intelligent machines to highly organized groups of people, which he calls “group
agents.” According to List's analysis, people who form groups can create new
centers of responsibility on the group level. Examples of this could be a
corporation (e.g., a tech company), an army, a club, or a religious organization.
If a group is able to organize itself in such a way that it has an internal structure
that allows it to make decisions, control the results of its actions, and come up
with collective moral judgments about its options, then there might be group‐
level moral responsibility on the part of the organization for what it does.
6.79 Each of the members of the group might of course be responsible human
beings in their own right. But the group itself—which is formed by these
responsible human beings—can also become an “agent” of its own, with its own
projects and its own responsibilities. Or so List argues. In fact, List even argues
that a group that behaves in a rational way can be seen as a form of “artificial
intelligence.” It is a human construction that is able to behave and make
decisions in intelligent ways. And such groups, List argues, can be responsible
for their actions and the outcomes of those actions.
6.80 In the same way, List continues, artificially intelligent machines might
come to have abilities that enable them to perform the sorts of actions and
functions that morally responsible human beings perform or are able to perform.
As List sees things, an AI system might be able to (1) exercise control over the
outputs of its behavior, (2) predict the likely effects of its actions, and (3) make
moral judgments about those actions and the outcomes of those actions. List
writes:
It should be clear that, while there are significant technological challenges
here, conceptually, there is no reason why an AI system could not qualify as a
moral agent and, in addition, satisfy the knowledge and control conditions
[for being a responsible agent]. Even if existing AI systems do not yet meet
these requirements, there is no reason to think that having an electronic or
otherwise engineered hardware is an in‐principle barrier to their satisfaction.
If we acknowledge that group agents can qualify as fit to be held responsible,
then we should be prepared to acknowledge that AI systems can do so too, at
least in principle. 11
(List 2021: p. 1229)
Daniel Tigard, who was already mentioned earlier, makes related claims in a
number of articles. He makes use of the idea that responsibility is a complex
notion, with many different “faces.”
6.81 When it comes to human beings, there are many different questions we can
ask about responsibility, which are related to the different faces of responsibility.
For example, if something good or bad happens, we can ask whether a certain
human being or group of human beings can be seen as having performed actions
or made decisions that helped to lead to the good or bad outcome. And we can
ask to what aspects of their personality or their patterns of thinking we can
attribute their behavior. This is sometimes called the attributability question
related to responsibility. We can also ask who (if anybody) is able to answer
questions about why something happened or why somebody acted in a given
way. This is the answerability face of responsibility. Additionally, we can also
ask who (if anyone) can be held to account for something that has happened or
something that somebody has done. This is the accountability face of
responsibility.
6.82 Most people who discuss these different “faces” or aspects of responsibility
tend to focus on human beings. But Tigard suggests that we can ask whether
robots and other machines might be able to, so to speak, take on some of these
different faces of responsibility. And Tigard is of the view that, yes, perhaps they
can. A robot, for example, might be viewed as being behind a certain outcome.
6.83 If an AI system, for example, is able to diagnose and then suggest a
treatment for some illness, and the treatment is then effective, then perhaps we
should attribute the success of this treatment to the AI system: this would be a
way of regarding the medical AI system as positively responsible for this
outcome. Perhaps this or other AI systems might also be able to answer
questions about why certain things happened or why it behaved in certain ways.
Suppose, for example, that it were possible to ask AlphaGo why it suggested
certain moves in the games of Go against Lee Sedol. And suppose that the
computer program AlphaGo was able to answer why it suggested these moves in
the game. AlphaGo, it might be said, could then be answerable for its suggested
moves, and perhaps responsible in that sense.
6.84 It might make less sense, Tigard grants, to hold AI systems, robots, and
other machines to account for what they do. So, they might not be able to be
responsible in the accountability sense of responsibility. But that is not the only
“face” of responsibility. And we have just seen that it might make sense to view
technologies as being responsible at least in the attributability and answerability
senses. At least, that is what Tigard suggests in some of his articles.
6.85 These suggestions from List and Tigard are fascinating. The idea of
machines that are responsible for what they do is both fascinating and, at the
same time, potentially a little bit scary to some people. But what should we think
about their suggestions upon reflection? Do these ideas make sense? And would
it be a good idea to have machines that we view as being responsible for what
they do?
6.86 The ideas that List presents raises questions that we will devote a whole
chapter to. List suggests that a machine can be a “moral agent,” i.e., an entity
that can make moral judgments and that can act on the basis of moral
considerations. This is an idea that some philosophers, computer scientists, and
others take so seriously that a whole field of research—“machine ethics”—has
been developed to explore this idea. For example, it is sometimes claimed—as
we saw in Chapter 3—that technologies such as self‐driving cars need to become
moral agents who sometimes make moral decisions, including life‐and‐death
decisions about what to do in accident scenarios when people's lives are at stake
in traffic. It is sometimes also claimed that future military technologies will have
to be moral agents who make decisions about what it is right for them to do in
war situations.
6.87 This is also, however, an idea that some philosophers and others argue
involves serious problems. As they see things, machines cannot be moral agents
of the sorts that human beings can be. Machines, according to this perspective,
lack certain abilities and features that we have as human beings, which enable us
to be moral agents who can be responsible for our own actions. This question—
i.e., whether machines can be moral agents who perform moral actions and make
moral decisions—is so fascinating that we will save it for later and devote the
whole next chapter to that issue.
6.88 Another possible worry about the idea that machines can be responsible is
that it might not be clear what the point would be of holding machines
responsible for what they do or for outcomes they produce. In other words, if
somebody is harmed or indeed is benefited by a machine and they are asked to
assign responsibility for this to the machine, one possible response one might
imagine that this person might have to this is “what's the point of that?” The
practice of holding people responsible for good and bad things should have some
sort of point. It might seem unclear whether there would be any clear point in
holding machines responsible.
6.89 Robert Sparrow, who worries about responsibility gaps related to military
robots, raises worries of this kind. Sparrow focuses on blame and punishment.
And he argues that the point of blaming or punishing somebody is that that
person should be made to feel guilty or ashamed about what they have done or
that they should suffer discomfort as they are being punished. A machine could
neither experience guilt or shame, Sparrow argues, nor suffer from any
punishments in the ways that human beings can do. And so there seems to be no
point, according to Sparrow, in blaming or punishing a machine.
6.90 If there is no point to blaming or punishing a machine, it would not make
sense, he concludes, to view the machine as responsible for what it is doing. We
can flip this around and focus on the positive side of things as well. When we
praise somebody and credit them with the responsibility for something positive,
the person in question will have occasion to feel pride and to be generally happy
about what they have done or helped to bring about. Since a machine can
presumably not feel proud or be happy about what it has done, there might not
seem to be any point in praising a machine—at least not like there can seem to
be a point in praising a human being.
6.91 We might not agree with the idea that the point of holding people
responsible for things is to make them feel good or bad. But we might
nevertheless agree with the general idea that there should be some clear point to
holding people (or machines!) responsible. And we might worry that holding
machines responsible is somewhat pointless. Moreover, it might also be claimed
that creating machines and other technologies and then holding those
technologies responsible for good or bad outcomes they produce is morally
questionable, as it could be seen as a form of human abnegation of responsibility.
6.92 Human beings, it could be argued, should take responsibility for the
technologies they produce and the outcomes of those technologies. They should
not “outsource” responsibility to their technologies. That, it might be thought, is
an irresponsible thing to do. So, we need to find a way, it could be argued, of
explaining how human beings can always be seen as being responsible, even
when the technologies that we create operate with some amount of autonomy
and even when these technologies possess artificial intelligence of different
kinds.
6.93 It is our general responsibility, it might be thought, to always take
responsibility for our creations and to be clear on how and why we can retain
responsibility. It might even be thought that there is something undignified about
creating technologies and then viewing ourselves as no longer being responsible
and instead handing over responsibility to these technologies. So, let us now
reflect on how human beings might after all be the ones who are responsible—
and who should take responsibility—for our technologies.
6.8 Human–Machine Teams and Responsibility
6.94 When people worry about responsibility gaps created by technologies that
we cannot directly control, whose inner workings we cannot fully understand, or
whose behavior we cannot fully predict, it might be that they envision these
technologies as acting completely on their own, completely outside of our
supervision and completely outside of our human control. If that were really the
case, then we should indeed really worry about responsibility gaps. And it would
then seem unfair to hold any human beings responsible for what these
technologies do. Nor could we justifiably take credit for any good outcomes that
these technologies produce. But are “autonomous” technologies—e.g., self‐
driving cars, military robots, or diagnosing systems—really completely
independent of human beings in this very strong sense? And would it be
attractive or desirable to create such technologies?
6.95 A better way of thinking about so‐called autonomous technologies, i.e.,
technologies that can operate for certain periods of time without direct human
steering, might be to think of them as being part of human–machine teams.
Related to this we can also think of human beings as performing the role of
managers or supervisors of the technologies that might be thought to give rise to
responsibility gaps. When we use technologies over time, we typically assess
their performance, update them in different ways, stop using them if we do not
assess their performance positively, start using them in combination with other
technologies, and so on. This is a way of exercising control over technologies,
even over ones that we do not directly control when they are in operation.
6.96 Perhaps a self‐driving car, for example, is getting around on its own without
direct human steering once it is up and running. But after the car has stopped, we
can assess how well it performed, whether we want to use it again, and whether
we might want to update it or add on other technologies to improve its
performance.
6.97 If we find that we cannot even indirectly control a technology, that we stop
understanding how it works, and that we cannot predict what it will do, but we
nevertheless continue using the technology again and again, then this might also
make us responsible for any problems the technology would cause. This might
be so even if we could perhaps not be seen as responsible for problems the
technology caused the first time that this happened and the first time that we lost
direct control over it.
6.98 If it becomes part of our regular practice that we continually use such
technologies, then we can be held responsible, because we have made it into a
practice of ours. It is a little bit like as if we start working with an unpredictable
and hard‐to‐control person. If we find that somebody who we are doing
something together with is unpredictable and that they cause problems, but we
nevertheless continue working together with that person, and we voluntarily
keep doing so, then we have to accept at least part of the responsibility for what
we are doing together with this unpredictable troublemaker.
6.99 The same could be said about unpredictable and hard‐to‐control
technologies that we keep “working” together with. If we immediately stop
using a technology that we find that we cannot predict or control or understand,
then we can perhaps justifiably deny responsibility for any bad unexpected
outcome that this technology caused. We might then be “off the hook,” as the
expression goes. But if we continue using the technology, well‐aware that it is
hard to control, predict, or understand, then this puts us back on the hook again.
6.100 In other words, it might make sense to make use of Christian List's
comparison between artificial intelligence and group agency, but to do so in a
different way. While List suggested that group agents (i.e., organizations that are
able to act in an intelligent way) is a form of artificial intelligence, we might
instead suggest that when human beings use technologies with a certain amount
of autonomy or with a certain amount of artificial intelligence, a new form of
group agent is created: namely, a human–technology team.
6.101 We form a type of group or organization together with a technology when
the technology is advanced enough that it can be seen as “acting” for, or together
with, us. And within a team, it is possible to ask whether there is somebody who
is in a position of responsibility: whether there is somebody who is “in charge,”
so to speak. Even if the technology is doing most of the grunt work, it can still
be the case—and, it could be argued, it should still be the case—that some of the
human beings involved are the ones who will have to take responsibility. This is
like when a military unit is acting in an organized way, and where the lower‐
ranked soldiers might need to do most of the “hard work,” but where the
commanders in charge are in a position of responsibility.
6.102 It is quite possible that we might completely lose control over some AI
technologies in the future. Authors like Nick Bostrom—who we already
encountered in a previous chapter—argue that at some point “super‐intelligent”
AI technologies will be created, and it will be impossible for us to control such
AI systems, even to control them indirectly. That might be so.
6.103 However, most of the technologies that people discuss when they discuss
potential responsibility gaps—e.g., self‐driving cars, AI‐driven computer
programs, or military robots—are not like the super‐intelligent AI systems that
Bostrom envisions in his work. We are able to stop using them, we are able to
update them, we are able to start using them in combination with other
technologies in order to achieve better results, and so on.
6.104 Insofar as the real‐world technologies we are familiar with today have
some degree of autonomy or some degree of artificial intelligence, they
nevertheless work together with us, work for us, and work under us. Human
beings are in charge in the kinds of human–technology teams that we have today
in the real world. So human beings can and should take responsibility for bad
outcomes caused by the technologies that we keep using even if we cannot
directly control, fully understand, or completely predict all of what these
technologies do when they are up and running.
6.105 It can seem less obvious, however, that human beings can in the same way
take credit for specific good outcomes created by advanced technologies. We can
end this chapter by returning to the example with which we started the chapter:
the victory over Lee Sedol by AlphaGo, where AlphaGo was “working together”
with a human being who was moving the stones around on the board, based on
the suggestions from AlphaGo. It certainly seems to be the case that the person
who was moving the stones around on the board cannot claim to be “in charge”
and therefore be mainly responsible in the positive sense for this victory.
Similarly, in many workplaces where people are instructed to use advanced
technologies and good outcomes are achieved, the people operating these
technologies might not be justified in taking a great deal of credit or positive
responsibility for the specific positive outcomes.
6.106 However, the organization as a whole might potentially be able to take
some credit for these good outcomes if at least two conditions obtain: (1) they
have developed or refined the technologies they are using and (2) they decide to
continue working with these technologies over time, based on the good
outcomes they are observing that working together with these technologies helps
to bring about.
6.9 Concluding This Chapter
6.107 The previous section is by no means intended to offer a complete solution
to worries we might have—and worries we might very justifiably have—about
potential gaps in responsibility that can be created by the advanced technologies
that we are developing in the contemporary world. The idea that we can think in
terms of human–technology and human–machine teams rather than in terms of
machines or technologies operating on their own is meant to be one idea about
how we can understand how human beings can still be responsible for good or
bad outcomes even when those outcomes are caused by advanced technologies.
It may be that most cases can be analyzed in such terms and that this can help to
fill many, if not most, apparent responsibility gaps. But it might at the same time
be that there are also other cases in which it is not so easy to think in these terms
and where serious worries about responsibility gaps remain.
6.108 This might be worries about all the four general kinds of responsibility
gaps considered above: backward‐looking positive responsibility gaps,
backward‐looking negative responsibility gaps, forward‐looking positive
responsibility gaps, and forward‐looking negative responsibility gaps. As with
all of the other chapters of this book, the aim of this chapter has been to
introduce these issues and to inspire the reader to reflect on—and to perhaps also
worry about!—these issues.
6.109 We will now move on to a closely related topic, raised by the discussion in
this chapter as well as the discussion in some of the previous chapters. Our next
topic will be the question of whether machines—or technologies more generally
—can be “moral agents” that make moral decisions and that act on the basis of
moral considerations and moral reasons. Is this even a possibility? If it is, is it
desirable to have machines that are moral agents, which might make morally
sensitive decisions about things that affect human beings? Might it be that
technologies could be moral agents, but that they should not be? These are
difficult and fascinating issues. So, without further ado, let us now jump into our
discussion of those questions.
Annotated Bibliography
Bradford, G. (2015). Achievement. Oxford: Oxford University Press.
A philosophical exploration of what it means for something to be a human
achievement.
Danaher, J. (2021). Robots and the Future of Retribution. In: Future Morality
(ed. D. Edmonds). Oxford: Oxford University Press.
Argues that perhaps it is not such bad thing if technologies create
responsibility gaps. Part of an accessible collection of essays about the future
of ethics.
Danaher, J. and Nyholm, S. (2021). Automation, Work and the Achievement
Gap. AI and Ethics 1 (3): 227–237.
Argues that automated technologies in workplace settings may give rise to
achievement gaps, thereby threatening people's opportunities for doing
meaningful work.
Hevelke, A. and Nida‐Rümelin, J. (2015). Responsibility for Crashes of
Autonomous Vehicles: An Ethical Analysis. Science and Engineering Ethics
21 (3): 619–630.
Argues that neither car manufacturers nor individual users should be held
responsible for crashes involving self‐driving cars—but that, instead, all users
of self‐driving cars should be held collectively responsible for any crashes
involving such technologies.
List, C. (2021). Group Agency and Artificial Intelligence. Philosophy &
Technology 34 (4): 1213–1242.
Compares the behavior of well‐organized groups of people to a form of
artificial intelligence, and argues that both organized groups of people and AI
systems can in principle be morally responsible for their actions and
decisions.
Maslen, H., Savulescu, J., and Hunt, C. (2019). Praiseworthiness and
Motivational Enhancement: ‘No Pain, No Praise’? Australasian Journal of
Philosophy 98 (2): 303–318.
Discusses the value of human effort.
Nyholm, S. (2020). Humans and Robots: Ethics, Agency, and
Anthropomorphism. London: Rowman & Littlefield International.
Discusses, among other things, whether robots are agents that can be morally
responsible. Chapter 3 argues that we should regard human beings as
supervisors or managers of robots and AI systems, and that human beings can
therefore be responsible for how technologies behave.
Sparrow, R. (2007). Killer Robots. Journal of Applied Philosophy 24 (1): 62–77.
Argues that autonomous weapons systems are unethical because they give
rise to responsibility gaps.
Talbert, M. (2019). Moral Responsibility. In: The Stanford Encyclopedia of
Philosophy (Winter 2019 Edition) (ed. E.N. Zalta).
https://plato.stanford.edu/archives/win2019/entries/moral‐responsibility.
A useful overview of the philosophy of moral responsibility.
Tigard, D. (2021). Artificial Moral Responsibility: How We Can and Cannot
Hold Machines Responsible. Cambridge Quarterly of Healthcare Ethics 30
(3): 435–447.
One of a series of articles by an author who explores whether machines can
be morally responsible.
Notes
1 For more information, read the Wikipedia entry “AlphaGo Versus Lee Sedol”:
https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol
2 For more discussion, see the science and technology studies expert Jack
Stilgoe's (2019) article “Who Killed Elaine Herzberg?” in OneZero here:
https://onezero.medium.com/who‐killed‐elaine‐herzberg‐ea01fb14fc5e
3 Readers who are curious about the philosophy of free will and moral
responsibility might find the interviews on a podcast called “The Free Will
Show” interesting: https://thefreewillshow.com
4 Check out the three episodes of the “Partially Examined Life” podcast
featuring Shoemaker as a guest here:
https://partiallyexaminedlife.com/tag/david‐shoemaker
5 For more information, see the Wikipedia entry “Death of Michael Jackson”:
https://en.wikipedia.org/wiki/Death_of_Michael_Jackson
6 See the website “Justice4cuz”: https://justice4cuz.com
7 For more information, see the section “Illness and Death” of the Wikipedia
entry “Prince (musician)”: https://en.wikipedia.org/wiki/Prince:
(musician)#Illness_and_ death
8 See “Artificial Intelligence: 6 Positive Visions for the Future of Work” on the
“World Economic Forum” website:
https://www.weforum.org/agenda/2021/11/positive‐artificial‐intelligence‐
visions‐for‐the‐future‐of‐work
9 You can hear Danaher speak about retribution gaps in this episode of the
“Philosophy 247” podcast: “Robots and Retribution”:
https://philosophy247.org/podcasts/robots‐and‐retribution
10 The philosopher Thaddeus Metz discusses a conception of reconciliation
informed by the African tradition in his video presentation “For
Reconciliatory Sentencing,” at the Royal Institute of Philosophy, here:
https://www.youtube.com/watch?v=TjmRMizvDfo
11 Here is a video in which List presents his view: “Group Agency and Artificial
Intelligence”: https://www.youtube.com/watch?v=xia6uGUvshU
7
CAN A MACHINE BE A MORAL AGENT?
SHOULD ANY MACHINES BE MORAL AGENTS?
7.1 Machine Ethics
7.1 In the previous chapter, we saw that Christian List claims that “it should be
clear that, while there are significant technological challenges here, conceptually,
there is no reason why an AI system could not qualify as a moral agent” (List
2021: p. 1229). As List sees things, AI systems might in principle even be able
to be morally responsible for their decisions. The kinds of examples List has in
mind are technologies acting in what he calls “high‐stakes settings.” That is to
say, List is talking about technologies—like self‐driving cars or military robots
—that have to make decisions that might have good or bad effects on human
beings. We can here also imagine machines making decisions that might have
good or bad effects on animals or on the natural environment. Such technologies
that are operating in high‐stakes settings need to make their decisions in a way
that is sensitive to moral considerations, List argues. This general idea—the idea
of technologies as moral agents—is the topic we will be exploring in this
chapter.
7.2 This is by no means an idea that is unique to List. In fact, there is a whole
field of interdisciplinary research called “machine ethics.”1 The aim of machine
ethics is to create what some authors call “artificial moral agents” or AMAs, as
this is sometimes abbreviated. This field of research is spearheaded by the
computer scientist Michael Anderson and the philosopher Susan Leigh
Anderson. Here is what Anderson and Anderson say about the ambitions of
machine ethics as a field:
The ultimate goal of machine ethics … is to create a machine that itself
follows an ideal ethical principle or set of principles; that is to say, it is guided
by this principle or those principles in decisions it makes about possible
courses of action it could take.
(Anderson & Anderson 2007: p. 15)
As we can see here, the ambition of those who work in the field of machine
ethics is not simply to create technologies that function in ways that are aligned
with human values (as in the general idea of AI value alignment). Rather, the
idea is to create machines that are able to recognize values and moral principles,
and that are able to act on the basis of those values and principles. As
philosophers sometimes put things, the machines these researchers imagine
would be able to act for “moral reasons.” They would do what they do because
they are able to discern moral reasons.
7.3 We have already extensively discussed one of the key examples that people
often focus on in this context: namely, self‐driving cars. As we have seen, self‐
driving cars might get into situations in which they need to make life‐and‐death
decisions. So self‐driving cars, some authors suggest, need to be sensitive to
moral reasons. Other examples of technologies that are sometimes said to need
to be able to be moral agents include autonomous weapons systems and other
kinds of military robots. They too will act in high‐stakes environments, where
important moral choices might need to be made.
7.4 There are also examples that are less dramatic, but nevertheless morally
sensitive. These include cases in which algorithms decide who gets a job
interview, who is fired, who gets a bank loan, who is likely to reoffend, and so
on. Those kinds of decisions may not be immediate life‐and‐death decisions, but
they affect people's lives. So it might be thought that technologies making such
decisions should also be sensitive to moral considerations. Anderson and
Anderson also consider technologies used in medical contexts, including care
robots. They interact with patients, and should be sensitive to the needs of these
patients and the values and principles governing proper medical care. Therefore,
such medical technologies also need to be moral agents, according to Anderson
and Anderson. There is no shortage, then, of technologies that are said to
potentially need to be moral agents that are able to act and make decisions on the
basis of moral reasons.
7.5 These ideas raise several questions. Here are some of them: is it really
possible to create machines that are moral agents? Can machines act or make
decisions on the basis of moral reasons? Is it necessary to create such machines
or can we avoid it? Would it be better to not create these machines, even if we
were able to? Should any machines be moral agents? Or should human beings
always be the only moral agents that make important moral decisions?
7.6 Those are the kinds of questions this chapter discusses. We will consider a
range of perspectives: both arguments in favor of machine ethics and arguments
against machine ethics. There is a lot of disagreement about whether it is
possible and also about whether it is a good idea to create technologies that are
moral agents. As we will see, some authors go so far as to argue that the field of
machine ethics could only produce the equivalent of “moral zombies” and
“psychopaths,” i.e., machines that behave as if they are moral agents but who
lack any conscious experiences or emotions of the sorts that are important for
moral agents to have.
7.7 Toward the end of the chapter, we will get to the issue of asking whether
machines can act for moral reasons is perhaps the wrong question to ask. In line
with what we discussed toward the end of the previous chapter, one perspective
we will explore again is whether it might be best to think in terms of human–
machine teams here.
7.8 That is, rather than only asking whether machines/technologies/robots can be
moral agents that can act on moral reasons on their own, we will also be
considering whether it might be better to ask whether human–machine teams can
be moral agents that can act and make decisions based on moral reasons. That is
where we will get on toward the end of this chapter. But we will begin by
focusing on the machines themselves, and ask whether they can or should
become moral agents in their own right. Let us jump right in.
7.2 Arguments in Favor of Machine Ethics and
Types of Artificial Moral Agents
7.9 As Anderson and Anderson see things, there are at least three strong reasons
to develop machines that are moral agents. The first reason we have just
considered in the introduction to this chapter above. It is that there are “ethical
ramifications” to what machines do. Machines and other technologies can help
or harm, or produce good or bad outcomes. So, ideally they should be sensitive
to ethical considerations. What about the other two reasons? The second reason
is that if we can build machines that are moral agents that are sensitive to ethical
considerations, then this might help to ease worries that people have about
autonomous machines that pose risks to human beings. The third reason that
Anderson and Anderson present is that they think that if we are able to figure out
how to create technologies that are moral agents, then this will help us to get a
better understanding of human ethics more generally. Creating ethical
technologies, in other words, might help to expand our understanding of human
ethics—or so Anderson and Anderson argue.
7.10 Two other researchers who have been influential in the development of the
field of machine ethics are Wendell Wallach and Colin Allen. They argue that
the creation of artificial moral agents is both “necessary” and “inevitable.” Their
idea here is that machines and autonomous technologies are increasingly being
introduced into various different sectors and domains of life: health care, elder
care, education, the military, transportation, social and intimate spheres, the
workplace, and so on. We would not want machines in these various domains,
Wallach and Allen argue, that are insensitive to ethical considerations. And with
the way technology is developing, it is unrealistic to think that we will be able to
stop the introduction of more and more advanced forms of technologies in all of
these and other domains of life. So the need to create ethical machines is pretty
much inevitable, Wallach and Allen argue.2
7.11 The computer scientist and ethics researcher Ronald Arkin also offers a
striking line of argument in favor of the machine ethics research project.3 Arkin
is particularly interested in the use of robots and other machines—such as
advising systems—in the context of warfare. He argues that there is a potential
for machines/technologies to become more ethical than human beings can be.
Later on we will see that some researchers object to the whole idea of machine
ethics because they think that machines cannot have emotions and because they
think that emotions are necessary for ethics. Arkin takes a very different view.
He argues that the lack of human emotions on the part of machines might enable
them to behave more ethically—for example in warzones—than humans.
7.12 We humans have complicated emotions, which can lead us to act
immorally. We are prone to emotional responses, such as anger, hatred,
cowardice, jealousy, envy, desires for revenge, and fear. Such emotions can drive
humans to commit war crimes. A machine that lacks such emotional tendencies
but that is able to reason about moral issues might become a superior type of
soldier, Arkin suggests. As Aimee van Wynsberghe and Scott Robbins (who we
will get back to later) helpfully summarize Arkin's view, he thinks that military
machines “would not rape or pillage the villages taken over during wartime and
would be programmed as ethical agents according to the Laws of Just War
and/or the Rules of Engagement” (Van Wynsberghe & Robbins 2018: p. 729).
Other researchers—including some other computer scientists and philosophers—
also defend this general line of argument. Machines could potentially be more
ideal moral agents than we human beings are able to be, these researchers argue.
So we should try to create artificial moral agents.4
7.13 This project can be more or less ambitious. To illustrate this, we can refer to
a classification from James Moor, who is one of the pioneers of the field of
“computer ethics,” a close relative to machine ethics. In an often‐cited text of
his, Moor distinguishes among different kinds of artificial moral agents that can
be envisioned. His distinctions are as follows.
7.14 An ethical impact agent is a technology that has ethically relevant impacts
on people, whether or not the technology is specifically built to be a moral agent.
This could be something as simple as a watch that is able to tell the time
correctly and that thereby enables people to keep promises by arriving to
meetings at their agreed‐upon times. Implicit ethical agents are technologies that
are designed with constraints that are meant to enable them to avoid creating
unethical outcomes. Such technologies might not explicitly be guided by ethical
principles in their decision making. In contrast, what Moor calls explicit ethical
agents would be technologies that are equipped with algorithms or other features
that enable them to explicitly make use of ethical considerations in their decision
making. Lastly, Moor also talks about full ethical agents. Those would be
machines or other technologies that are ethical agents in the same ways in which
we human beings can be ethical agents, with all of what this might be taken to
involve. For example, that might be a type of machine that would have a free
will or that would be conscious or whatever is taken to be distinctive of our
human form of moral agency.
7.15 When we are considering arguments for and against machine ethics, it can
be useful to think carefully about what kind of moral agents we are talking
about. Are we, for example, for or against machines as full moral agents? Or are
we for or against machines as implicit or explicit moral agents? Probably we are
not going to be too concerned with discussing whether or not to create ethical
impact agents. Yet, it is worth keeping in mind that most technologies are going
to be what Moor calls ethical impact agents. When we use technologies, this
almost always has potential ethical implications. Even so, when researchers
debate the pros and cons of machine ethics, they usually mean technologies that
are ethical agents in one of the more advanced senses: i.e., ones that are thought
to potentially be explicit moral agents or perhaps even full moral agents. More
on this later. For now, let us consider some important critical responses to
machine ethics.
7.3 Objections to the Machine Ethics Project
7.16 There are those who are very critical of machine ethics and the reasons that
have been offered in favor of creating artificial moral agents. Two authors, for
example, who are highly suspicious not only of whether it is possible to create
artificial moral agents but who are also highly critical about whether there are
any good reasons to create them are Aimee van Wynsberghe and Scott Robbins,
who were already briefly mentioned earlier.
7.17 Contrary to Wallach and Allen, van Wynsberghe and Robbins think that
creating machines that engage in ethical decision making is neither necessary
nor inevitable. We do not need such machines, they argue. What we need are
simply machines that are “safe” for humans to be around. According to their
perspective, trying to create artificial moral agency is a way of trying to escape
responsibility for our own actions. We should not try to create moral machines
and then outsource responsibility for what happens to these machines. Instead
we should try to create safe machines and safe technologies more generally,
which benefit and do not harm human beings. This is an example of the kind of
critical reactions that machine ethics is sometimes met with.5
7.18 In this section, we will consider four different objections that have been
raised against the machine ethics project. This section will be sub‐divided into
sub‐sections so that the four different objections are easy to find. In the next
main section after this one, we will consider possible responses to the objections
discussed in this section.
7.3.1 First Objection: Morality Cannot Be Fully Codified
7.19 Among the authors who are critical of machine ethics and the idea of
creating artificial moral agents are the American philosophers Duncan Purves,
Ryan Jenkins, and Brian Talbot. This team of authors offer several objections,
some of which we will get to below, but let us here consider the first objection of
the two main objections they have to machine ethics. They argue that machine
ethics researchers tend to assume that being ethical amounts to mechanically
following certain rules, whereas in real life, ethical behavior and decision
making cannot be easily codified to a set of simple and clear rules. So machine
ethics rests on an oversimplified and therefore mistaken conception of ethics.
7.20 Ethics is contextually sensitive. Being an ethical agent requires exercising
the capacity for judgment. According to Purves, Jenkins, and Talbot, machines
lack this capacity for judgment, which is necessary to make context‐sensitive
ethical judgments. So it is not possible to create machines that are moral agents
(at least not moral agents of a very sophisticated kind).
7.21 Importantly, this objection does not mean to deny that there can be general
ethical principles or that the types of general ethical theories discussed in
Chapter 2 might tell us a lot about what ethics is. It is possible to take the view
that Purves and company take while at the same time believing that general
ethical principles described by theories such as utilitarianism, Kantian ethics,
Confucian ethics, ubuntu ethics, and so on are valid general principles of ethics.
The point is that those are very general ethical ideals. Relating the general values
and principles within such ethical theories to particular situations and the choice
among different options for action is not always straightforward. It requires the
exercise of the capacity for context‐sensitive judgments that human beings have
but that technologies may lack.
7.22 There are some philosophers—the so‐called “ethical particularists”—who
believe that there are no valid general ethical principles. According to them, each
situation needs to be assessed ethically in light of its particular features, and no
general ethical principles are useful in real life.6 That, if correct, might seem to
be an even stronger objection to the ideal of creating moral machines. But again,
that is not the objection that Purves and company are making. They are simply
arguing that going from general ethical ideals to ethically correct responses to
particular situations requires a form of context‐sensitivity and a capacity for
judgment that they do not believe is possible to create in machines.
7.3.2 Second Objection: It Is Unethical to Create Machines
that We Allow to Make Life‐and‐Death Decisions About
Human Beings
7.23 Several authors find the idea that machines would be allowed to make
ethically sensitive decisions about human beings (or about animals) to be in
itself ethically problematic. This could be thought to apply to ethically sensitive
decisions more generally. But it is often thought to be a particularly strong
objection when it comes to life‐and‐death decisions—that is, decisions about
who should live or die. According to authors who present this objection, it is
deeply unethical to allow machines to make such decisions, and perhaps even
more unethical to allow the machines to then carry out those decisions by killing
human beings.
7.24 Two of the examples that have been discussed above are often discussed in
this context: namely, the examples of self‐driving cars and of military
robots/autonomous weapons systems. Interestingly, authors who discuss this
objection differ in whether they think that it applies to all machines, or only a
sub‐set of them. And authors who discuss this objection also differ in how they
spell out more fully what they have in mind.
7.25 Consider first the view of Purves and his co‐authors, for example. They
think that it is morally permissible to let self‐driving cars operate in high‐stakes
settings where they might occasionally kill people. Not only that; they think that
it can be morally permissible for those self‐driving cars to kill people according
to certain moral principles in unavoidable accident scenarios. Recall the kinds of
cases discussed in Chapter 3—e.g., when a self‐driving car can either go left and
hit five or go right and hit one person. In such situations, it could be justified,
according to Purves and company, for the car to be designed to respond in
certain ways, such as by minimizing overall harm, which might need to mean
killing the smaller number of people.
7.26 Yet, according to Purves et al., it is immoral to create and use autonomous
weapons systems that might kill people. What's the difference here? The
difference, they think, is that the autonomous weapons systems would have as
their main goal or main function to be “killer robots,” which are meant to kill
human beings in wartime situations. That is what they are specifically designed
to do. Self‐driving cars, in contrast, would have as their main function to allow
people to travel from A to B in as safe a way as possible. It is merely a foreseen,
but not directly intended, side effect that there will occasionally be accidents, in
some of which people might be killed by these technologies. The difference in
what these technologies are intended to do makes a big moral difference,
according to Purves et al.
7.27 Others, such as Nassim JafariNaimi, think that it is also unethical to create
self‐driving cars that will sometimes make life‐and‐death decisions about who
should live or who should be killed in an accident scenario. We already looked at
some of her reasoning about this in Chapter 3—as well as some related
arguments from John Harris. There is something instrumentalizing and
something not fully responsive to the value and dignity of human life to think of
traffic accidents involving self‐driving cars on the model of the trolley problem,
JafariNaimi thinks. Self‐driving cars should not be making decisions about who
lives or dies. Similarly, as we also saw in Chapter 3, Harris thinks that this puts
machines into the role of judges who punish certain people by “condemning
them to death.” That is something that machines should never do, according to
Harris. Decisions about life and death should always be made by humans, as
Harris sees things. So self‐driving cars or other technologies should not make
such decisions.
7.28 Robert Sparrow, who we have also mentioned before, also opposes life‐and‐
death decision making on the part of machines—in particular, “killer robots”—
because he thinks this involves gaps in responsibility and because he thinks that
no decision to kill a human being should ever be made unless there is somebody
who can take full responsibility for this decision. Not having a responsible
person who can take on that responsibility is incompatible with respect for
human beings and human life, Sparrow thinks.
7.29 Similarly, the team behind the campaign to “stop killer robots” writes the
following on their website:
People, not machines, must be held accountable. But if people are not making
meaningful decisions, then they cannot properly be considered responsible
for the consequences of their actions. It would be unjust to make a person
liable for the actions of an autonomous weapon system beyond their effective
control. If we are committed to accountability, then we need rules that ensure
that the right people are taking responsibility in the use of force.7
In summary, this second objection is that it is wrong to allow machines to make
ethically sensitive decisions. It can be wrong, for example, because this does not
respect the human dignity of those who have to suffer the consequences of these
decisions, or because there might be responsibility gaps. The project of machine
ethics is, among other things, the project of creating machines that make
ethically sensitive decisions, including life‐and‐death decisions. So, the whole
machine ethics project—according to this objection—is unethical.
7.3.3 Third Objection: Moral Agents Need to Have Moral
Emotions and Machines Do Not/Cannot Have Emotions
7.30 Speaking of the campaign to stop killer robots, here is another objection
that they used to feature on their website against building autonomous weapons
systems that we allow to make life‐and‐death decisions:
Fully autonomous weapons would decide who lives and dies, without further
human intervention, which crosses a moral threshold. As machines, they
would lack the inherently human characteristics such as compassion that are
necessary to make complex ethical choices.
(quoted in Nyholm 2020: p. 154)
In more recent versions of their arguments against killer robots on their website,
they have slightly altered this text. Instead of talking about machines as lacking
“human characteristics such as compassion that are necessary to make complex
ethical choices,” the text now says that machines lack understanding:
machines cannot make complex ethical choices, they cannot comprehend the
value of human life. Machines don't understand context or consequences:
understanding is a human capability…8
Here in this sub‐section, we will focus on the idea that machines lack emotions
and that they therefore cannot and should not make ethical choices. But I am
also quoting the other bit above as well, to illustrate a more general idea that we
will also be discussing in the next sub‐section: namely, the idea that machines
are not able to be sensitive to values in the way that humans are. As this claim is
often put, machines “cannot act for reasons.” (What that claim means or might
be taken to mean, we will discuss below.)
7.31 Regarding emotions in particular, it is a common idea that ethics is not only
about cold calculations and rational thinking. Ethics is also about being able to
have and understand emotions. There is a range of the so‐called moral emotions
—e.g., shame, guilt, resentment, compassion, sympathy, etc.—that are a key part
of human ethics. Since machines lack these emotions, this objection continues,
they lack direct access to an essential part of human ethics.
7.32 The philosopher Mark Coeckelbergh discusses this idea in striking terms. In
one of his articles, he discusses the question of whether robots can “be moral,”
as he puts it. The argument, as he understands it, goes as follows. The argument
begins with a partly descriptive and partly normative premise. According to that
premise, a moral agent both needs to have and should have certain emotions.
The second premise of the argument is that robots (and other machines) lack
emotions—because they do not have the kinds of mental states associated with
emotions. Robots are, in a way, like “psychopaths,” Coeckelbergh suggests. That
is to say, they are like individuals who lack the capacity for the typical sorts of
social emotions that regular moral agents have. The conclusion of this argument
is that for these reasons, robots (or machines more generally) cannot be moral.
7.33 The descriptive part of the argument Coeckelbergh discusses states that
emotions are as a matter of fact involved in the sorts of responses and social
interactions that we typically associate with human moral agents. The normative
part of the argument postulates that certain human emotions ought also to be
involved in morally acceptable interpersonal relations and social interaction. For
example, we typically expect somebody who has acted wrongly to feel guilty or
to feel ashamed of their conduct. This is a descriptive statement about what we
often expect of moral agents. However, we also think that a person who has
acted wrongly should feel guilty or perhaps should be ashamed of their conduct.
This is a normative idea. So, to repeat, if a robot or any other machine lacks the
capacity for the relevant kinds of emotions, it would fail to satisfy both the
descriptive and normative conditions on being a moral agent related to emotions.
7.34 Briefly put, then, machines—at least any current machines—are like
“psychopaths” in the sense that they lack the types of emotions that we associate
with human agents. We saw earlier that Arkin and some other machine ethics
enthusiasts think that this might be a positive thing, since emotions sometimes
lead people to act in bad ways. However, others who are critical of machine
ethics think that their lack of emotions make machines unfit to play the role of
moral agents. Having and thereby being able to understand emotions is,
according to this objection, a crucial part of being a moral agent. So, we should
not make machines that are supposed to be moral agents. Or so this third
objection claims.
7.3.4 Fourth Objection: Machines Are Not Able to Act for
Reasons
7.35 We will now consider another objection that is discussed by Purves and
colleagues, among others. They—and many others along with them—think that
being ethical requires the capacity to act on the basis of reasons. Moreover, they
think that only human beings have the capacity to act on the basis of reasons.
Therefore, only human beings can be moral agents, according to this objection to
machine ethics.
7.36 As Purves and colleagues see things, there are two main theories of what it
is for a person to be “acting for reasons.” And they think that both of these are
out of reach for machines. According to one theory, acting for reasons amounts
to acting on the basis of certain beliefs and desires. According to a second
theory, acting for a reason amounts to acting on the basis of a distinctive type of
mental state, which we can describe as “taking something to be a reason.” As
human beings, we have minds, and within our minds, we can have beliefs and
desires, and we can have the distinctive mental state of perceiving something as
being a reason to act. But Purves and coauthors argue that machines do not have
minds with these kinds of mental states, because they lack any significant kinds
of minds at all. Hence machines cannot act for reasons. So, if being a moral
agent requires the capacity to act for reasons, machines cannot be moral agents
—or at least that is the idea behind this objection to machine ethics.
7.37 Carissa Véliz also discusses this general type of objection to the idea that
technologies can be moral agents, and she does so in very striking terms. Véliz
focuses on algorithms in particular. She compares algorithms—even very
“smart” algorithms—to zombies. Here is what she says about what she means by
the word “zombie”:
Zombies … lack conscious experience. It is nothing there is like to be a
zombie—they do not suffer from cold when it is cold, they do not feel
sadness when they cry, and philosophical puzzles do not worry or excite
them.
(Véliz 2021: p. 487)
Zombies, as Véliz understands them in her philosophical argument, lack what
she calls “sentience.” This term refers to the capacity to have feelings and
conscious experiences—such as the ones that Véliz mentions in the examples in
the quote above.
7.38 Véliz argues that any entities that are zombies cannot be moral agents that
are able to respond properly to moral reasons. This might be the zombies that
look like human beings that we are familiar with from horror movies. Or it might
be technologies that are zombies in the metaphorical sense of lacking any
sentient minds. The key consideration that Véliz thinks speaks in favor of this is
that moral reasons often have to do with an understanding and first personal
appreciation of the sorts of conscious experiences that are so often very relevant
from an ethical point of view, such as the capacity for suffering or other forms of
felt experiences.
7.39 Véliz also argues that we should reject the view that Moor puts forward on
which we can distinguish between “explicit moral agents” that act on moral
principles and “full moral agents,” who can act on principles and moral reasons
in the way that human beings do. Véliz rejects that idea because she thinks that
acting on principles or moral reasons in the way that we human beings do is the
only possible way to act on moral principles and reasons. At least that is so when
it comes to moral reasons that have to do with ethically important forms of
conscious experiences. Hence Véliz writes that “explicit ethical agents and full
ethical agents are one and the same category” and that “we will not get an agent
that can identify ethical problems and respond adequately to them without
sentience” (Véliz 2021: p. 489).
7.40 Véliz further develops her argument by writing the following, which is
worth quoting at length:
A reason, for moral agents, amounts to something that matters to us, that we
care about because we can understand its moral significance. Algorithms are
programmed to do something: win a game of chess, distinguish spam from
non‐spam, identify people who might want to buy a product, assess whether a
candidate will be appropriate for a job, etc… . When such algorithms perform
their tasks, they are not wondering whether it is morally correct to prey on
vulnerable people … We need sentiments, passions, to be motivated to act
morally … If algorithms do not have access to subjective experiences that are
attached to value, then they will lack moral motivation because they will be
incapable of appreciating moral reasons.
(Véliz 2021: p. 494)
Véliz's argument here is a little bit like a mix of the argument Coeckelbergh
discusses (i.e., the one according to which machines are like “psychopaths”
without emotions) and the argument that Purves and colleagues present (i.e., the
one according to which machines cannot act for reasons because they lack the
right sorts of mental states). The argument can be summarized as follows. If an
entity is a zombie that lacks emotions and other conscious experiences, this
entity cannot understand and fully appreciate moral reasons that are related to
ethically important conscious experiences. Such entities cannot be moral agents.
Algorithms and other technologies are like “zombies,” which lack emotions and
experiences. Therefore, Véliz concludes, algorithms and other technologies
cannot be moral agents.9
7.3.5 Brief Reminder of the Objections to Machine Ethics
Considered Above
7.41 Below, we will consider possible ways of responding to the objections to
the machine ethics project—i.e., the project of creating machines that can be
moral agents that make moral decisions and perform moral actions. But let us
first have a brief reminder of what the four objections raised above were.
7.42 The first one was that machine ethics seems to assume that ethics can be
completely codified into simple rules, and that these rules can then be
programmed into machines in an unproblematic and straightforward way. The
problem with this idea is that ethics is contextually sensitive, and that there
might not always be straightforward answers to how general ethical ideals relate
to specific ethical challenges in particular situations.
7.43 The second objection was that there might often be something ethically
problematic about allowing machines to make ethically sensitive decisions. This
might especially be the case if the decisions in question are life‐and‐death
decisions about who should live or die. Such decisions—if and when they should
be made—should be made by human beings. It can seem to offend against
human dignity to allow machines to make ethically sensitive decisions about us
human beings.
7.44 The third objection was that ethics importantly involves emotions. On the
one hand, moral agents are thought to have certain moral emotions. On the other
hand, moral agents ought to have certain moral emotions in certain
circumstances. So, if machines do not or cannot have emotions, this emotional
aspect of ethics cannot be captured by artificial moral agents. That can seem like
a problem. Moral machines can seem to be a form of “psychopaths.”
7.45 The fourth objection was that ethics requires the capacity to act on the basis
of moral reasons. According to one interpretation of this objection, acting for
reasons requires conscious mental states that artificial moral agents would lack.
According to another interpretation of this objection, being able to appreciate
moral reasons requires being able to consciously experience pain and other
morally relevant conscious states. Moral machines can seem to be a form of
“zombies,” who lack the conscious states that can seem important to have in
order to respond to and understand moral reasons.
7.4 Possible Ways of Responding to the
Critiques of the Machine Ethics Project
7.46 How might one respond to these objections? That is the question we will
now consider. One possible response is to try to defend the machine ethics
project. Another possible response is to try to reimagine what the machine ethics
project should try to achieve. According to this second kind of response, we
should not try to build artificial moral agents that are moral agents in the ways in
which we human beings are moral agents. Instead, we should try to build
machines and other technologies that could be some other form of moral agent,
which could work together with human beings—or assist human beings—in
their ethical decision making and moral agency. Let us now consider both kinds
of responses, the first one more briefly and the second in some more detail. The
second type of response we will divide into two different parts, in Sections 7.4.2
and 7.4.3.
7.4.1 First Response: Bottom‐Up Learning Rather Than
Top‐Down Rule‐Following
7.47 One response that one sometimes sees to the objection that machine ethics
is problematic because ethics is not about simple rules that we follow
mechanically is to suggest that we should not aim to build rule‐following
machines that mechanically respond to ethical situations based on simple rules.
Rather, we should make use of machine learning techniques to allow AI systems
to uncover deeper patterns that are part of human ethics. Different forms of
machine learning can be used. Parts of ethics that are hard to codify into very
simple and clear rules can be discovered by the artificial intelligence in the
technologies we create. Or so this response would go.
7.48 Why might one make such a suggestion? One reason is that one might think
that there are certain very complex underlying principles of ethics that we human
beings somehow are able to intuitively learn and understand but that we find it
hard to articulate—especially into simple rules.
7.49 There is the idea—associated with the famous linguist and philosopher
Noam Chomsky—that human language has an underlying universal grammar
that we all have an intuitive ability to grasp, but that is very hard to articulate.
That idea has been picked up by some ethics researchers, who argue that human
ethics might also involve certain complex underlying principles that are hard for
us to capture in simple rules. There is, according to researchers such as John
Mikhail, an underlying “universal moral grammar.”10
7.50 This might be something, it could be argued, that AI systems could help us
to articulate and get clear on. And those AI systems might then themselves start
operating on the basis of these complex and hard‐to‐articulate moral principles
underlying human ethics. That is one possible suggestion.
7.51 Another idea that some different researchers have articulated and defended
is that we should use virtue ethics as a model for the machine ethics project.
According to different forms of virtue ethics—both “Western” virtue ethics like
the virtue ethics in the Aristotelian tradition and “non‐Western” virtue ethics,
like the Confucian virtue ethics tradition—tend to emphasize learning and
practice as part of human moral development. In the same way, some authors
argue that machines should not be created as fully formed moral agents. Rather,
machines could also learn from experience. They could develop virtues through
a type of bottom‐up process that would have similarities with the process by
which human beings learn to become virtuous.
7.52 This idea would involve that machines would both try to learn from good
exemplars—virtuous people that they could emulate—and that they could learn
by practicing virtue. In the human case, Aristotle, Confucius, and others have the
idea that we can learn from more experienced and wiser human beings and try to
emulate them. And there is also the idea that one learns to become fully virtuous
by first acting in virtuous ways, trying to make this into a habit, and then—in the
process—internalizing these habits and thereby developing true virtues. Perhaps
machines with machine‐learning capabilities could do the same. That is the idea
at least, in this response to the first objection we considered above.
7.53 If that response would be (partly) successful as a response to the first
objection considered above, that would still leave us with the other three
objections—and perhaps other objections to machine ethics as well. In order to
consider how those other objections might be countered, let us consider a more
radical type of response to the objections.
7.54 That type of response would not be to try to defend the project of building
machines that are a form of independent moral agents. It would instead be to
question the wisdom of trying to do so. The approach here might be to think
about whether technologies could somehow support human ethical decision‐
making or perhaps offer an alternative to it. Or it could be to consider whether
some machines could, so to speak, act together with human beings, so as to form
“human‐technology teams,” which could be moral agents of an important kind.
Let us try to get there via an idea about whether partial moral agency is a much
more realistic goal than full moral agency in machines.
7.4.2 Second Response: Resisting the Idea That
Machines/Technologies Should Ever Be Full Moral Agents
7.55 We saw above that Véliz argues that there cannot be such a thing as an
explicit moral agent that is not at the same time a full moral agent of the sort that
we human beings are. To be able to explicitly follow moral rules and act on
moral reasons, one needs to be able—Véliz argues—to consciously experience
and understand morally relevant ideas and considerations, including morally
relevant feelings and sensations. Let us think more generally about whether this
is a good strategy within technology design and in the creation of artificial
intelligence in particular. Should technologies—and AI technologies in
particular—aim to do things in maximally humanlike ways?
7.56 Sometimes it can make sense and also be interesting to try to create
technologies that are humanlike and that do their tasks in humanlike ways. But
more often than not, technologies work better if they carry out tasks in
alternative ways, which are different from how human beings—with our human
bodies and human brains—do things. Machines/technologies will typically do
things in different ways than humans. And they might achieve better results in
this way than if they are meant to do things like human beings do them.
7.57 Moreover, many forms of human agency and human responses are highly
complex, containing many different parts. Emotions, for example, are often
thought to involve various different changes in the person who enters into an
emotional state. Emotions involve things such as changes in patterns of thought
and motivation, changes in conscious feelings, changes in facial expressions,
changes in heartrate, changes in what we pay attention to, and so on. Perhaps a
machine could replicate some—or perhaps even many—of the different aspects
of human emotions, but not all of them. So, they could, we could say, partially
have emotions, but not completely have human emotions. Machines might not
be able—at least not yet—to have the conscious feelings associated with human
emotions, for example. But other aspects of human emotions might be replicable
in machines.
7.58 So, two things seem important to note here: first, machines might perform
better if they (at least in part) do things in different ways than human beings.
And second, machines may often or at least sometimes be able to replicate parts
—perhaps many parts—of complex human responses. Perhaps we should think
like this in the domain of ethics as well.
7.59 That is to say, we could say that machines should handle ethical decision‐
making in a different way, or slightly different way, than human beings handle
ethical decision‐making. And second, we might say that machines could and
should be able to replicate many parts of human ethical decision making, but that
it would be unrealistic—and perhaps not even desirable—to try to completely
replicate all aspects of human ethical decision making or human ethical behavior
in machines.
7.60 The machine ethics project, according to this way of thinking, should be
about creating an alternative form of ethical agents. It should perhaps recreate
some aspects of human ethics. But artificial moral agents could—and perhaps
should—be a different kind of moral agents than we human beings are.11
7.4.3 Third Response: Switching to Thinking in Terms of
Human–Machine Teams Rather Than in Terms of
Independent Artificial Moral Agents
7.61 If we start thinking about artificial moral agents as doing things in partly
different ways than human moral agents—and we start thinking of them as
replicating parts but not all aspects of human moral agency—the question arises
of whether it might not make sense to think of machines and humans as
complementing each other in the domain of ethical decision making. For
example, perhaps rather than asking whether machines can act on the basis of
moral reasons, we can ask whether human–machine teams can act on the basis of
moral reasons.
7.62 Let us consider the example of military robots—which is the example that
most opponents of machine ethics worry the most about and object most strongly
to. Should we be asking whether military robots can be moral agents of their
own, of some independent form? And should we be asking whether such
independent artificial moral agents can understand reasons and act on the basis
of reasons? Here, it can seem to make a lot of sense to instead ask whether
military robots can be part of human–machine teams. Military robots can “work
together” with human soldiers. The human beings and the military robots can
form a team. If human teams can understand and act for reasons, perhaps
human–machine teams could act for reasons—including moral reasons—as well.
7.63 Recall the ideas from Christian List discussed in the previous chapter. List
claims that human beings can form organized teams that can be “group agents,”
which can act in morally responsible ways. How does that happen? It happens
when there are procedures by which the human beings in the team make
decisions together and carry out projects together. When human beings organize
themselves and act together in such ways, they can become capable of doing
things that no single human being would be able to do on their own.12
7.64 In the same way, humans can interact with technologies—e.g., military
robots—in organized ways, so that they can pursue certain goals and make
decisions according to some procedure. Human–machine teams might then be
able to do things that no human being could—or at least could not easily do—on
their own. Such human–machine teams could then perhaps be said to be able to
act on the basis of reasons. And this might include moral reasons.
7.65 It might be correct, as Véliz argues, that being able to act on reasons
requires the ability to have conscious experiences of certain kinds. But it might
be that the whole agent, so to speak, does not need to be able to have conscious
experiences. A “group agent” (i.e., a human organization of the sort that List
discusses in his work) could have members who are able to understand and
experience certain things in conscious ways. The group agent formed by the
different people working together might lack any conscious states. But the group
agent might still appreciate moral reasons because some or all of the members of
the group agent—i.e., the people who are part of the organization—might be
able to have the relevant conscious experiences.
7.66 In the same way, it might be seen as being enough that the human beings
who work together with certain technologies (again, perhaps military robots)
would be able to have conscious experiences. The group that is formed by
certain humans and certain robots might not have conscious experiences on the
group level. But the human–machine team might be able to act on the basis of
moral reasons because the humans involved can appreciate and understand the
conscious states that are relevant to be able to experience and understand.
7.67 This idea could also be related to the worry that moral machines would be
like “psychopaths” without the emotions usually associated with ethics. That
worry relates to the idea of machines considered as independent moral agents on
their own. And the worry is that they would lack the sorts of emotions that moral
agents can have and sometimes should have. But if we think of machines as
being parts of human–machine teams that can be a form of moral agents—where
tasks are divided between the humans and machines—then this worry might be
eased, at least in part.
7.68 The human beings who are part of these human–machine teams can have
the relevant moral emotions. The “job” of the machines involved might be to
carry out certain calculations or to make certain recommendations or to perform
certain physical tasks. The job of the humans involved in the human–machine
teams might, in part, be to have or be able to have some of the emotions that are
important in ethical situations. The humans in the human–machine teams might
have other tasks as well. But one key task might be to lend the “human touch” to
these teams. And this might be done in part by having capacities for emotions
that some of the robots or other technologies involved might not have.
7.69 Lastly, it could be claimed that while there might be something highly
morally questionable about letting machines make ethically important decisions
—including life‐and‐death decisions—about human beings on their own, it
might be less problematic to let human–machine teams make ethically sensitive
decisions. Again, this might be less problematic since the humans involved in
these human–machine teams might have the conscious sensitivity—including
emotions—that the machines in the human–machine teams would lack.
7.70 If important decisions about our lives are made by machines alone, this can
seem like a clear blow to our human dignity. However, if human beings make
use of machines—or if they, so to speak, “team up” with certain machines—
when they make ethically sensitive decisions about us, then this might be less
like a clear blow to our human dignity. Again, the humans involved can help to
add the necessary “human touch.” The humans involved can also be accountable
for what the human–machine teams are doing. So, there might not be any
responsibility gaps of the sorts that Sparrow and others worry about.13
7.5 Concluding This Chapter
7.71 David Mindell is a historian of technology and also an engineer who has
been involved in the development of autonomous submarines that are able to
explore deep‐sea ocean floors. In his 2015 book Our Robots, Ourselves, Mindell
describes the history of autonomation in a number of different areas up until that
point in time: for example, automation of airplanes with aviation, automation of
spaceships within space exploration, the just‐mentioned deep‐sea exploring
submarines, military robots, self‐driving cars, and so on. An interesting
observation at the center of that book is that up until now, in many of these just‐
mentioned fields of engineering where autonomy and artificial intelligence have
been explored, the following development can be noted. Early on, many
engineers have often thought that full autonomy would get the best results and
be most convenient. However, again and again, it has seemed that the best results
are achieved when humans and machines work together.
7.72 For example, deep‐sea exploration has been more efficient and has achieved
better results when the submarines have been semiautonomous and where people
have been involved in the work of those technologies. Of course, things might
change in the future. But for now, Mindell's hypothesis is that for a long time,
human–machine teams will typically perform better than fully autonomous
machines operating on their own (something that Mindell thinks is mostly a
fantasy at the current time anyway) or human beings on their own, without the
help of modern technologies.14
7.73 Perhaps we should apply Mindell's reasoning to the idea of moral machines
as well. The idea of machines making moral decisions and performing moral
actions on their own, we could argue, is likely to be a myth or a fantasy—at least
in the foreseeable future. Moreover, even if machines become “better” in some
sense at making moral decisions and performing moral actions than human
beings are, it might still seem ethically problematic to outsource moral decision
making and moral action to machines that we build.
7.74 If we argue in this way, would we be abandoning the aims of the machine
ethics project, as Anderson and Anderson or Wallach and Allen conceive of it?
On some interpretations of that project, perhaps. However, it might be that the
machine ethics project was never intended as a project to create wholly
independent moral machines that would be a sort of unique individuals, like we
consider people (or mature adults) to be.
7.75 Then again, we might also question the idea of the wholly independent
human agent, who makes his or her decision wholly on their own, completely
independently of others, and whose thinking is completely uninfluenced by other
people or the group(s) to which he or she belongs. We are, as human beings, part
of what might be called “ethical communities”—what are sometimes called
“moral tribes,” by those who take a somewhat more cynical attitude toward this
idea. We influence each other and we discuss ethical issues and we come up with
ethical norms within society, some of which are also made into laws. So, if
machines enter into the moral community, perhaps it should be only natural to
think of them as joining certain groups of people, rather than as being wholly
independent moral decision makers in their own right.
7.76 If machines in some way or other enter the moral community as moral
agents that help to make moral decisions and that help us to perform moral
actions, one question that arises is whether they could also be members of the
moral community in the sense of being what philosophers call “moral patients.”
That is to say, if machines—e.g., robots—become members of the moral
community in some sense, is it possible that there could be ways in which we
could treat them rightly or wrongly? Should they ever be considered as having a
moral status of some sort? That is the question we will now explore in the next
chapter.
Annotated Bibliography
Anderson, M. and Anderson, S.L. (2007). Machine Ethics: Creating an Ethical
Intelligent Agent. AI Magazine 28 (4): 15–26.
A clear statement of the aims behind the project of trying to create ethical
machines.
Arkin, R. (2009). Governing Lethal Behavior in Autonomous Robots. Boca
Raton, FL: CRC Press.
Argues that machines might be able to be more ethical in the battlefield than
human soldiers are able to be.
Coeckelbergh, M. (2010). Moral Appearances: Emotions, Robots, and Human
Morality. Ethics and Information Technology 12 (3): 235–241.
Discusses whether having emotions is necessary in order for somebody or
something to be a moral agent.
List, C. (2021). Group Agency and Artificial Intelligence. Philosophy &
Technology 34 (4): 1213–1242.
Compares the behavior of well‐organized groups of people to a form of
artificial intelligence, and argues that both organized groups of people and AI
systems can in principle be morally responsible for their actions and
decisions.
Mindell, D. (2015). Our Robots, Ourselves: Robotics and the Myths of
Autonomy. New York: Viking.
Discusses the history of automation and argues that human‐machine teams
will typically perform better than automated machines on their own.
Moor, J. (2006). The Nature, Importance, and Difficulty of Machine Ethics.
IEEE Intelligent Systems 21 (4): 18–21.
An oft‐cited article that distinguishes among different types of more or less
sophisticated forms of artificial moral agents.
Purves, D., Jenkins, R., and Strawser, B.J. (2015). Autonomous Machines, Moral
Judgment, and Acting for the Right Reasons. Ethical Theory and Moral
Practice 18 (4): 851–872.
Argues, among other things, that machines cannot be moral agents because
they cannot understand moral reasons.
Van Wynsberghe, A. and Robbins, S. (2018). Critiquing the Reasons for Making
Artificial Moral Agents. Science and Engineering Ethics 25 (3): 719–735.
A spirited criticism of the arguments offered in favor of the machine ethics
project.
Véliz, C. (2021). Moral Zombies: Why Algorithms Are Not Moral Agents. AI
and Ethics 36: 487–497.
Argues that machines that lack consciousness cannot be moral agents.
Wallach, W. and Allen, C. (2009). Moral Machines: Teaching Robots Right from
Wrong. Oxford: Oxford University Press.
A relatively early—but still highly relevant—discussion of the idea of
creating ethical machines.
Notes
1 “The Machine Ethics Podcast” is devoted to the machine ethics project, and
Susan Anderson and Michael Anderson, who we are just about to get to, are
among the guests who have been interviewed on that podcast:
https://www.machine‐ethics.net. Here is the episode featuring Anderson and
Anderson: “15. Machine Ethics With Susan and Michael Anderson”:
https://www.machine‐ethics.net/podcast/15‐susan‐and‐michael‐anderson
2 You can hear Wallach speak on the topic of this chapter in the video
presentation “Moral Machines: From Machine Ethics to Value Alignment”:
https://www.youtube.com/watch?v=AcRDuH4Qtnw
3 Here is an interview with Arkin, from the “Interaction Hour” radio show: “The
Interaction Hour: Don't Call Them Killer Robots, With Dr. Ron Arkin”:
https://www.youtube.com/watch?v=ax_‐uPzWmyw
4 For a much less optimistic vision of what the future of autonomous weapons
might involve, watch the 2017 “Slaughterbots” video from the Future of Life
Institute: “Slaughterbots” https://www.youtube.com/watch?
v=HipTO_7mUOw. Here is a sequel, from 2021: “Slaughterbots—If Human:
Kill ()”: https://www.youtube.com/watch?v=9rDo1QxI260
5 You can hear Robbins speak about this topic in his video presentation “What
Machines Shouldn't Do,” here: https://www.youtube.com/watch?
v=xvgbYhNQHSg
6 Here you can hear the philosopher Jonathan Dancy discuss particularism on
the “Philosophy Bites” podcast: “Jonathan Dancy on Moral Particularism”:
https://philosophybites.com/2012/06/jonathan‐dancy‐on‐moral‐
particularism.html
7 “Stop Killer Robots”: https://www.stopkillerrobots.org/stop‐killer‐robots/facts‐
about‐autonomous‐weapons (Accessed on December 16, 2021).
8 See the “Facts about Autonomous Weapons” section of the “Stop Killer
Robots” website: https://www.stopkillerrobots.org/stop‐killer‐robots/facts‐
about‐ autonomous‐weapons
9 Here is a video in which David Chalmers discusses zombies from a
philosophical point of view: “Are There Philosophical Zombies? David
Chalmers for the Royal Institute of Philosophy”:
https://www.youtube.com/watch?v=‐UTlcF‐OT8o
10 You can listen to Mikhail discuss the idea of a universal moral grammar on
the “Philosophy Bites” podcast here: “John Mikhail on Universal Moral
Grammar”: https://philosophybites.com/2011/06/john‐mikhail‐on‐universal‐
moral‐grammar.html
11 In this video, Pim Haselager discusses differences between human
intelligence and artificial intelligence. He also argues that instead super‐
intelligence, what we need is super‐ethics: “Super Ethics or Super
Intelligence—Pim Haselager”: https://www.youtube.com/watch?
v=RDDoOIcgzE4
12 Here is an episode of “Philosophy Bites” in which Philip Pettit explains the
idea of group agency in a clear way: “Philip Pettit on Group Agency”:
https://philosophybites.com/2010/12/philip‐pettit‐on‐group‐agency.html. In
the podcast, Pettit talks about ideas from a book about that topic that he wrote
together with Christian List.
13 In this episode of the “Zero Pressure” podcast, you can hear Britain's first
astronaut, Helen Sharman, discuss the idea of human‐machine teaming with
her guests Captain Michael Brasser, commander of the 59th task force of the
US Navy, and the air traffic management expert Sameer Alam: “Human‐
Machine Teaming and Evolution: What Is Next?”:
https://www.youtube.com/watch?v=DoHYWCpqxrU
14 You can hear Mindell speak about his book Our Robots, Ourselves in this
video: “Our Robots, Ourselves|David Mindell|Talks at Google”:
https://www.youtube.com/watch?v=4nDdqGUMdAY
8
CAN ROBOTS BE MORAL PATIENTS, WITH
MORAL STATUS?
8.1 The Tesla Bot and Erica the Robot
8.1 In August 2021, the technology entrepreneur and CEO of the car company
Tesla, Elon Musk, presented his plans—or what he said were his plans—for the
“Tesla Bot.” It was not altogether clear whether the presentation was meant to be
taken seriously, given its content. But on the other hand, Musk did not say
anything that suggested that he was not being serious. During this presentation,
Musk said that the self‐driving cars that Tesla has developed “are basically semi‐
conscious robots on wheels” and that as a company that is developing self‐
driving cars, “Tesla is really the world's largest robotics company.” Hence it
makes sense, Musk continued, to give these supposedly semi‐conscious robots
that Tesla has developed a “humanoid form.” Hence the idea for the Tesla Bot.1
8.2 Musk showed a picture of a robot with a humanoid form—though with no
facial features and no humanlike colors, but which was instead partly black and
partly white in color. There was even a person dressed in a full body suit of the
same colors, who was performing a dance routine before Musk started talking.
But once Musk started talking, that dancing illustration of what the robot would
look like left the stage. And Musk started referring to pictures in a slide show
instead.
8.3 Musk explained some of the technologies he envisioned that the Tesla Bot
would be equipped with: both artificial intelligence (AI)‐related technologies
and the more mechanical types of technologies that these robots would be using.
Musk also explained what purpose he thought that these robots could serve.
They could relieve human beings of undesirable work, by eliminating
“dangerous, repetitive, boring tasks”—seemingly a take on the more commonly
used phrase that robots can take over “dull, dirty, and dangerous tasks” from
human beings.
8.4 For anybody who might find the prospect of these robots uncomfortable or
who might be worried that we might lose control over them, Musk was quick to
assure viewers that the robots would be designed to be “friendly.” They would
only be motivated to serve and help human beings. Moreover, Musk said that the
robots would be weak enough that one could easily win in a fight against them.
And in addition to that, they would be slow enough that one could easily run
away from a Tesla Bot if one would become frightened.
8.5 The vision of the Tesla Bot, then, was a vision of a faceless, vaguely
humanlike robot that would serve us, that would have no will of its own, and that
we could easily overpower if necessary. In a way, that can seem to fit with
Joanna Bryson's normative version of the instrumental theory of technology that
we were introduced to in Chapter 1: i.e., the idea that any robot or other
technology should be a mere means, and never an end in itself, but instead
always a tool that serves human beings. In Bryson's stark phrase (which she has
since said that she has regretted using), “robots should be slaves.” This seems to
fit the idea behind the “humanoid robots” that Elon Musk said that he wanted to
build during his August 2021 presentation.
8.6 A very different vision of a humanoid robot comes from the Japanese
robotics researcher Hiroshi Ishiguro. In addition to creating a robotic copy of
himself, which looks remarkably similar to him, Ishiguro has also created
various other humanoid robots, including one that looks like a Japanese woman.
That robot is called “Erica.” And on one occasion, Ishiguro said that it (or she)
was the most beautiful woman he has ever seen. These robots are not faceless,
and they are not created to be mere tools that we can overpower and treat in a
purely instrumentalizing way. Rather, Ishiguro interacts with his robots—
perhaps especially with Erica—in a way that almost shows a degree of reverence
toward the robots. They are treated as persons, not as mere tools or things. They
are treated in what are sometimes called anthropomorphizing ways. That is, they
are treated as if they do not only look like human beings but as if they also have
other humanlike qualities.2
8.7 This tendency to treat robots—and perhaps especially humanoid robots—as
if they are persons or at least more than mere tools can be observed in the way
that many people interact with robots. One famous example, which has already
been brought up in earlier chapters, is the robot Sophia, from the company
Hanson Robotics. When the former chancellor of Germany, Angela Merkel, met
Sophia, for example, she posed for a “selfie” photograph together with Sophia.
One would not do such a thing with, say, a hammer, a frying pan, a camera, or a
screwdriver. Such technologies are treated like mere tools normally. But many
robots—again, especially ones resembling humans—tend to provoke
anthropomorphizing responses in many people.
8.8 Such treatment can be of a seemingly positive kind—e.g., taking a selfie
photo with a robot, like Merkel did with Sophia. That is the sort of thing one
might do with a friend or with somebody one admires. And this is not the only
example of Sophia the robot being treated in a positive way. Notably, as was also
mentioned in Chapter 1, Sophia was even given “honorary citizenship” of the
Kingdom of Saudi Arabia at a technology and futurism event in 2017. That also
seems like a clear example of what might be dubbed a way of treating a robot
“well,” so to speak. But anthropomorphizing treatment of robots can also be of a
more sinister or at least ethically questionable sort.
8.9 This can be exemplified with the help of the sex robot Roxxxy, which was
created by the American inventor Douglas Hines. When Hines presented
Roxxxy at a technology event in 2010, he explained that he had equipped this
sex robot with different personality settings or different modes of behavior.3
Some of these were of the sort that one would expect in a sex robot. For
example, the robot would engage in sexual banter and respond favorably to
different kinds of touch. However, the robot also had a more controversial
personality mode: the “Frigid Farah” setting. In this mode, the robot would not
welcome the sexual advances of the user, but would rather say “no” and behave
as if it did not wish or consent to any sexual interaction. This was seen as wrong
by some commentators, who claimed that this robot would enable people to
enact rape fantasies—a form of behavior that they judged to be wrong.
8.10 When one reflects on these kinds of examples, one question that comes to
mind for many people is whether robots should ever be given rights or whether
they should ever be treated with moral consideration, like we think that people
(and animals) should have rights and be treated with moral consideration. In the
more technical language that philosophers sometimes use, we can ask whether
robots—especially humanoid robots—could ever be “moral patients.”
8.11 A moral patient is any entity toward which it is possible to act rightly or
wrongly: any entity toward which we could have obligations. Could robots or
any other technologies—for example, avatars or characters in computer games
or in other virtual worlds—ever be moral patients? Would it be desirable to
create technologies that could be moral patients? Or would it be best to avoid
doing so, as Bryson thinks? Is it perhaps morally problematic to even start
thinking about whether robots and other technologies should be treated with
moral consideration or be given rights? Is that a distraction from more important
issues? And does it perhaps show disregard for those—such as humans and
animals—who are most clearly moral patients? These are the kinds of questions
this chapter will be discussing.
8.12 The chapter will have a special focus on the kinds of robots in the examples
above. That is, the chapter will focus on humanoid robots: robots made to look
and/or behave like human beings. The chapter will focus on humanoid robots for
three reasons. First, these are the kinds of robots where the question of whether
robots or other technologies could ever be or become moral patients seems most
realistic or most interesting. Second, in a lot of science fiction, humanoid—or at
least vaguely humanoid—robots tend to be portrayed as moral patients. When
we enjoy science fiction, many of us often have no problem imagining robots as
some form of moral equals or near moral equals alongside human beings. Third,
there are people in the tech world, such as Elon Musk, Hiroshi Ishiguro, David
Hanson, and Ben Goertzel at Hanson Robotics, who are clearly fascinated with
the idea of humanoid robots and who are developing them.
8.13 So even if we do not yet have many such robots that we interact with in our
lives, and they mostly seem like science fiction at the present, it might be that a
few more years into the future, there will be humanoid robots that we will be
confronted with on a fairly regular basis. Before that time, it may be a good idea
to reflect on whether such robots—or at least some of them—should ever be
treated with some measure of moral consideration.
8.2 What Is a Humanoid Robot? And Why Would
Anybody Want to Create a Humanoid Robot?
8.14 In the 1920 stage play (which premiered in January 1921) that introduced
the word “robot”—Rossum's Universal Robots—by the Czech playwright Karel
Čapek, the robots in the play are a little bit similar to the Tesla Bot that Musk
wanted to create a little over 100 years later in 2021.4 That is, those robots also
look like artificial humans. And they are also built to work for human beings, in
their case in a factory. In fact, the word “robot” derives from the Czech language
word “robota” which means something along the lines of forced labor.
8.15 In a way, then, Musk's vision stays true to the original idea of a robot, from
the play in which that term was used for the first time. In real life, though, most
robots that are of use to human beings do not look like human beings, not in the
present at least. Think, for example, of the Roomba vacuum cleaning robot,
which some or many readers of this book might have at home or at least be
familiar with. Or think of the robots that are used in factories to build cars,
robots that are used to explore the ocean floor, or military robots. None of those
robots look like human beings. Instead, they have shapes that are relevant to the
tasks they are meant to perform. Roomba looks a little like a hockey puck or a
beetle, for example. Robots “working” in warehouses sometimes look like boxes
or containers on wheels, which can carry packages around. Some military robots
look like small tanks.
8.16 By the expression “humanoid robot,” as noted above, this chapter means to
refer to a robot that has a humanoid appearance and that might perhaps also be
made to behave like a human being. A maximally humanlike humanoid robot
would be one that both looks and acts in ways that are indistinguishable from a
real human being. Such robots are not very realistic at present. But there are
humanoid robots in existence or in development, which look like human beings
and that behave like human beings, at least in some limited ways. Let us
consider some examples.
8.17 A sex robot, for instance, is an example of a humanoid robot. Roxxxy,
which was mentioned before, is a sex robot that is created to look like a human
woman and that can perform certain simple movements and that has a basic chat
function, a little like a chatbot such as Alexa or Siri. Sex robots are created to be
artificial sex partners, but as we will discuss in the next chapter, some of them
are also created to be companions.
8.18 Sophia the robot is another example of a humanoid robot. That particular
robot does not have any obvious function—other than, perhaps, being a way of
getting people interested in robots and, in particular, the robots created by
Hanson Robotics. Sophia is a controversial robot. It has been called a deceptive
“show robot” by critics, such as Noel Sharkey—a roboticist who has turned into
a robot ethicist, and who often criticizes those who want to build humanoid
robots.5
8.19 Other examples of humanoid robots include Kaspar, a robot made to look
like a child, which is used in experimental therapy for children with autism.6
That robot is supposed to help children with autism to open up to social
interaction with strangers. So it seems important that that type of robot would
have a humanlike form, since that is part of the strategy behind the therapy. It is
supposed to display a simplified humanlike appearance and behavior that is less
overwhelming to children with autism than more complex, real human beings
can be to these children. This humanoid robot is supposed to serve as a kind of
“gateway,” so to speak, to the more complex interaction that these children
might come to have with human beings who are strangers to them.
8.20 Speaking of the idea of humanoid robots as a form of therapy assistants, the
same idea has also been suggested in relation to sex robots. Neil McArthur—a
philosopher who has devoted much of his research to the philosophy and ethics
of sex—has suggested that sex robots might help people who have experienced
sexual traumas to get back into the practice of sexual interaction with others. Just
like children with autism might find interaction with people they do not know
overwhelming, some people who have had traumatic sexual experiences (e.g.,
sexual assault or rape) might be able to get comfortable with the idea of having
sex again by first having sexual interactions with a humanlike robot that they can
have control over. Or so McArthur suggests in an article defending the idea of
creating and using sex robots.
8.21 Notably, some people think that there are reasons to be skeptical about the
project of creating humanoid robots. One such reason that is often discussed is
that there would be something uncanny or otherwise freaky or uncomfortable
about robots that look and behave like human beings. People sometimes speak
about the so‐called uncanny valley hypothesis. This is the idea that when robots
start looking and behaving like humans but still are clearly different from real
human beings, many human beings would experience interaction with such
robots as somehow eerie, unsettling, or strange.
8.22 Interestingly, Alan Turing already made claims along those lines in a radio
speech he recorded for the BBC back in 1951. Turing, as noted in a previous
chapter, was a pioneer of computer science and the field of AI—somebody
deeply interested in creating machines that could think or behave as if they could
think. But Turing was at the same time uncomfortable with and skeptical about
the idea of creating humanoid robots. Here is a quote from the radio broadcast
Turing recorded for the BBC (the audio of which is unfortunately no longer
available!):
… I certainly hope and believe that no great efforts will be put into making
machines with the most distinctively human, but non‐intellectual
characteristics such as the shape of the human body. It appears to me to be
quite futile to make such attempts and their results would have something like
the unpleasant quality of artificial flowers. Attempts to produce a thinking
machine seem to me to be in a different category … I believe that the attempt
to make a thinking machine will help us greatly in finding out how we think
ourselves.7
For Turing, then, there was an important difference between wanting to create a
machine that could think (or behave like it could think, which is how he thought
about AI), on the one hand, and wanting to create machines that look and act like
humans, on the other hand. The former, Turing thought, could help us to better
understand how we ourselves think. But the latter would have something
unpleasant about it—a not very attractive form of artificiality.
8.23 It is fascinating to compare what Turing says in the quote above with the
reasons that are sometimes offered why it can be a good idea to try to create
humanoid robots. Hiroshi Ishiguro, for example, says that one of the reasons to
try to create humanoid robots is that it can help us to understand ourselves better.
This can be compared to the idea that we encountered in the previous chapter,
where we saw that machine ethics researchers such as Anderson and Anderson
think that we can come to understand human ethics better if we manage to build
ethical machines.
8.24 The creators of Sophia the robot from Hanson Robotics say similar things.
David Hanson has even published a book entitled Humanizing Robots: How
Making Humanoids Can Make Us More Human. The two opening paragraphs
from that book are worth quoting in full, given how they contrast and compare
with the quote from Turing above. Hanson writes:
The human face is the dynamic icon of the human identity. It embodies our
sense of self and others, not just in concept but in our very neural
infrastructure. The face seizes attention. The history of art, artifact, and
entertainment proves our persistent fascination with the human likeness.
Sometimes realistic, other times fantastic; sometimes beautiful, other times
hideous—always the face speaks to us. It is the primary input‐output device
of the human species. Thus, the humanlike robotic face could be one of the
most promising new paradigms for computer interfaces—if the nuances of
the human face can be mastered in robotic media …
(Hanson 2017: p. 1)
Hanson continues:
Humanlike robots reflect not just our faces but also our thoughts. They allow
us to explore the deep aspects about what makes us human by simulating
human cognition, by testing people's cognitive responses to the robots, and by
challenging the human identity in robotic artworks. The cognitive aspects of
the robots are perhaps the most conceptually interesting aspects of humanlike
robots. Yet they also promise to be of profound importance to our future.
Humanoids push artificial intelligence (AI) toward human‐level, “strong” AI.
(Hanson 2017: p. 1)
While Turing found the idea of humanoid robots unpleasant and thought it futile
to attempt to create such robots, though he thought that creating thinking
machines could help us to understand ourselves better—Hanson thinks the
project of creating humanoid robots could have a whole host of potential
benefits. It could be a form of artwork. It could help us understand ourselves
better. It could make us more human. And it could help us to develop human‐
level AI.
8.25 To summarize this section: the idea of creating humanoid robots—i.e.,
robots that look and behave like human beings—both repels and fascinates
people. It is an idea that is a common trope in science fiction. The idea of robots,
as noted in a previous chapter, comes from science fiction, rather than from
science. But robots and even humanoid robots have gone from not only being
part of science fiction to also being part of science and real life. Since they are
polarizing—since they seem like a terrible idea to some, but a great idea to
others—let us now start considering whether there might be reasons to treat any
robots and humanoid robots in particular with some degree of moral
consideration. Can they be moral patients? Can one act rightly or wrongly
toward them?
8.3 Can People Act Rightly or Wrongly Toward
Robots?
8.26 The first thing to notice when it comes to whether it is possible to act
rightly or wrongly toward robots is that people at least sometimes seem to judge
that we can do so. This can be illustrated with the following example from 2015,
which we already had occasion to mention in Chapter 1. As was noted in
Chapter 1, in that year, CNN Edition ran a story with the headline “Is it Cruel to
Kick a Robot Dog?”8 This was not a story about a humanoid robot but about a
four‐legged robot that was nicknamed “Spot,” and that resembled a dog when it
moved around.
8.27 The robot was created by the company Boston Dynamics. In addition to
conjuring up the idea of a dog, this robot also had the distinguishing feature that
it is very good at keeping its balance. To illustrate this, a video released by
Boston Dynamics shows Spot walking up some stairs and running on a
treadmill. Later in the video, however, and again in order to illustrate how stable
Spot is, a Boston Dynamics engineer is shown kicking Spot. Sure enough, Spot
does not fall over when kicked. Many viewers of this video, however, lost their
composure when they saw Spot being kicked. CNN reported that these viewers
made comments such as “Kicking a dog, even a robot dog, just seems wrong”
and “Poor Spot!”
8.28 This was an example of people seemingly making moral judgments about
how a robot is treated—ethical judgments that indicate that they were viewing
this robot as a moral patient toward which it was possible to act in good or bad
ways. Imagine now that we were not dealing with a robot dog, but instead a
robot that looked like a human being. For example, imagine that somebody was
kicking, not Spot the robot dog, but Erica the robot, i.e., a robot that looks like a
human woman. If people react with disapproval toward the sight of engineers
kicking a robot dog in order to show how stable it is, it is likely that many would
react even more strongly to the sight of people kicking a robot that is made to
look and behave like a human being.
8.29 This might especially be so if a robot is made to look like a particular
person. Take, for example, not Erica but rather the other robot created by Hiroshi
Ishiguro mentioned above: namely, the robotic copy of himself that Ishiguro has
created. Surely Ishiguro might feel that there is something wrong—perhaps that
he is being attacked in some way—if somebody would start kicking or otherwise
attacking the robotic copy of him.
8.30 In that case, the robotic copy was made by Ishiguro himself, or at least he
was deeply involved in the design of this robot. (Perhaps he built it together with
other people; or perhaps they built it based on his specifications.) It might seem
even worse if somebody else would build a robotic copy of us, and then treat that
robot in what would seem like a bad way, e.g., by kicking or hitting it or by
treating it in some other undignified way.
8.31 In fact, it might also seem strange and uncomfortable to us if somebody
created a robot copy of us and then wanted to have that robot copy of us as their
new best friend. There is something generally uncomfortable about the idea of
others wanting to create copies of us (robotic or otherwise), independently of
whether they treat those copies well or badly.
8.32 Does this suggest that the robots themselves should have their own
personalities and not be copies of any actual human beings? Does it perhaps
mean that we should not create humanoid robots to begin with? But if we do
create them, would they ever be—or could they have features that could make
them into—moral patients? The reader might here wonder whether this is a topic
that technology ethics researchers have taken seriously and written about. If they
have, is that a worthwhile topic to devote one's time to thinking and writing
about? Where would one even start when trying to think about this? Many
questions arise when one starts thinking about this issue.
8.33 David Gunkel—who is the author of such books as Robot Rights and How
to Survive a Robot Invasion—is somebody who has written extensively about
whether robots should ever be treated with moral consideration. And he has
studied the existing literature on this topic in great detail. In addition to the very
detailed overview of the literature that existed on this topic already by 2018,
which is when Gunkel's book Robot Rights came out, Gunkel has also been
continually updating a map or visual representation of the landscape of different
views that have been explored and defended by philosophers, computer
scientists, and others about whether robots could be moral patients. The map was
already complex and full of different views—some of them radically opposed to
each other—back in 2018. With every new update of the map of the field that
Gunkel posts online, the map is getting more and more intricate.9 More and more
positions are being staked out in this debate. So it can seem a little
overwhelming and hard to know where to start when we think about whether
robots could or should ever be seen as moral patients toward which we can act
rightly or wrongly.
8.34 Here are three questions that we can use to create some order in this chaos
of different philosophical views and arguments about whether robots can be
moral patients:
Can robots have morally relevant properties or abilities?
Can robots imitate or simulate morally relevant properties or abilities?
Can robots represent or symbolize morally relevant properties or abilities?
Many of the key contributions that have been made to the discussion of whether
robots can be moral patients can be understood as being explorations of—or can
be classified with the help of—these three questions.
8.35 As noted above, there are also those who think that we should not even be
seriously discussing this topic in the first place. We will get to that idea later on.
Let us first start by discussing the three questions above. And let us relate some
of the existing literature to these three questions that can be asked about robots
as potential moral patients.
8.4 Can Robots Have Morally Relevant
Properties or Abilities?
8.36 Above, we considered whether it might be wrong to kick a robot dog. One
of the things that one can do here is to ask whether the moral status of animals
can be a model for the potential moral status of robots.10 To get to that
comparison, we can first reflect on what it is about animals that lead people to
think that they should be seen as moral patients with an important moral status.
As we saw in Chapter 2, the utilitarian school of thought is one type of ethical
theory that is relevant to consider here. So let us return to some points that were
brought up in Chapter 2. We can start with Bentham.
8.37 When Jeremy Bentham, the well‐known utilitarian philosopher of the 18th
century, discussed and defended his utilitarian ethical theory, one of the most
famous lines that appears in a book of his is about whether animals should be
thought to have moral status. Bentham's line goes like this: “the question is not,
Can they reason?, nor Can they talk? But, Can they suffer?” Animals can suffer,
Bentham continued. And because he thought the ability to suffer was what
counted in ethics, Bentham thought that we should include animals in the class
of moral patients toward which we can act rightly or wrongly. That is, according
to Bentham, animals have an important moral status that we should take into
account from an ethical point of view.
8.38 At the same time, however, Bentham thought that human beings had a more
important moral status than animals do. This is because humans can suffer in
different ways than animals are able to. Bentham writes:
we must consider that man is not like the animals, limited to the present,
whether as respects suffering or enjoyment; but that he is susceptible of pains
and pleasure by anticipation; and that it is not enough to secure him from
actual loss, but it is necessary also to guarantee him, as far as possible,
against future loss …11
On this type of view, then, the capacity to suffer and to experience happiness is
what grounds moral status or—in other words—makes one into a moral patient.
Animals can suffer from what they presently experience. So they have moral
status, as Bentham sees things. But human beings can also anticipate future
pleasures and pains—and they can also think back on past pleasures and pains—
and so they have a more advanced form of moral status than animals.
8.39 The earlier quote about how we should not ask whether animals can reason
or talk but about whether they can suffer alludes to another common theory of
what grounds moral status. Namely, the theory that moral status has to do with
whether one is a rational being who can think, talk, and reason. This is Kant's
perspective. Kant emphasizes that human beings are persons with reason and the
capacity to act morally. This is what makes us into ends‐in‐themselves (as Kant
puts things), as opposed to mere means. Our human moral status, therefore, is
based on our ability to think, to reason, and to act on moral principles.
8.40 If animals had these abilities, then Kant would think that they would be
moral patients with an important moral status as well. In fact, Kant speculates in
some of his books that there might be aliens on other planets who are rational
and who can act ethically just like we can. And Kant thinks that they would have
moral status just like we do as well.
8.41 Let us now consider how all of this relates to robots and other technologies.
The American philosopher Eric Schwitzgebel and the philosopher and artist
Mara Garza have written a couple of papers where they offer the following
argument. They first argue that we should treat like cases alike. In other words,
just like Kant claims that any aliens with rationality would be moral patients or
Bentham claims that any animals which can suffer are moral patients, we should
apply the same standards to non‐human entities as we apply to human beings.
Schwitzgebel and Garza then suggest that in theory, there could be AI systems or
other technologies that would have the sorts of properties or abilities that we
associate with our human moral status. They then conclude that any such
possible AI systems or other technologies, if they existed, would have the same
moral status as human beings do. Or if their abilities were similar to those of
animals, then the AI systems or those other technologies would have the same
moral status as animals. For example, if we could create robots that are able to
suffer, that can have social relationships, or that can think rationally, then
Schwitzgebel and Garza think that we should treat those robots (or whatever
technologies it would be) with the same amount of moral consideration that we
treat human beings with.
8.42 Interestingly, Joanna Bryson, who thinks that we should treat robots and
other technologies as mere means and not as ends, agrees with Schwitzgebel and
Garza. Bryson thinks that it is in theory possible to create robots and other
technologies that would be moral patients to whom we would have certain
obligations. However, as Bryson sees things, it is a moral imperative to avoid
creating any such machines. We should only create machines that can justifiably
be treated as mere tools. It is unethical to create machines to which we would be
obligated and which we could not treat as mere tools.
8.43 A similar argument is also made by the German philosopher Thomas
Metzinger. Metzinger studies consciousness in his research, and he has become
convinced that it is in theory possible to create machines that would be able to
feel pain. However, Metzinger thinks that it would be unethical to create such
machines, since they would suffer. So we should not create any machines with
feelings, Metzinger argues.12
8.44 Those views are radically different from those we looked at in the previous
chapter when we discussed the topic of whether machines can be moral agents.
Many philosophers, such as Carissa Véliz, think that technologies lack and will
probably always lack consciousness and any subjective experience. Technologies
—and algorithms in particular—are like “moral zombies,” Véliz thinks, as we
saw in the previous chapter. Véliz discusses that idea in relation to whether
technologies can be moral agents. But presumably Véliz would use the same
argument as a basis for concluding that machines can also not be moral patients.
8.45 It is worth pointing out here that there is a lot of disagreement about
whether it is possible to create machines that have some form of consciousness.
It might be safe to say that the view that Véliz takes—i.e., that machines lack
consciousness and that they are unlikely to become conscious anytime soon—is
the standard view among many if not most researchers. However, there are also
many researchers—such as Bryson (a computer scientist and ethics researcher),
Metzinger (a philosopher), and others—who think that it is possible to create
conscious machines, as we just saw.
8.46 Bryson even suggests in one of her articles that some existing machines
have a limited form of consciousness. If by consciousness we mean the ability to
take in information and process that information in one's “head,” and then being
able to report one's perceptions to other agents, then some machines can be said
to have a basic form of consciousness, according to Bryson.
8.47 There are also those who are actively trying to develop robots that can feel
pleasure and pain. For example, the Japanese robotics researcher Minoru Asada
is engaged in the project of trying to build robots that can feel pleasure and pain.
His theory is that this can help to create a new form of AI. The idea is that since
human beings partly learn by experiencing pleasure and pain (and this is
certainly the case before we become able to talk and understand language while
we are babies), machines that can feel pleasure and pain will be able to learn in
the way that human babies learn.13
8.48 To summarize this section: one question we can ask is whether robots and
other technologies can have the sorts of properties and abilities that matter from
a moral point of view. For example, we can ask whether any robots or other
machines would be able to suffer, to think, or to reason or to act on ethical
principles—all of which are the sorts of properties and abilities that are often
thought to ground moral status so that those with those abilities should be treated
with moral consideration.
8.49 There is disagreement about whether robots or any other technologies can
have or will ever have the kinds of abilities that would give them moral status of
an important kind. It is often thought, for example, that being conscious or
having the ability to be conscious is an ethically important property that gives
people and animals moral status. And it is controversial whether robots could be
or ever become conscious in the ways that humans and animals are.
8.50 So, if the issue is whether robots can have morally important properties and
abilities, it might be unclear what we should think about this. But as noted
above, there are also other important questions that can be asked when we are
reflecting on whether it is possible to act rightly or wrongly when we are
interacting with robots and other machines. Let us now consider the second
above‐mentioned question that it can be interesting to reflect on.
8.5 Can Robots Imitate or Simulate Morally
Relevant Properties or Abilities?
8.51 Let us return to Turing's ideas again. Recall that Turing thinks that rather
than asking whether machines can think, it is better to ask whether machines can
behave in such a way that people would perceive them as if they are able to
think. If machines that we have created are able to imitate intelligent human
beings or human thinking, then we have successfully created intelligent
machines, as Turing sees things. What matters, according to Turing, is how
machines behave, not whether they have conscious experiences or whether they
have minds that are similar to human minds.
8.52 John Danaher—the Irish legal academic and philosopher we have already
encountered in some previous chapters—thinks that we should apply Turing's
reasoning when we think about whether machines can be moral patients. That is,
Danaher thinks that Turing has a good test for whether machines are intelligent
—the test of how they behave and whether they can imitate intelligent behavior.
And Danaher thinks that we should use a similar test—an ethical Turing test, if
you will—when we think about whether we should regard machines as moral
patients toward which we can act rightly or wrongly.
8.53 Danaher notes that Turing's view can be regarded as a form of
“methodological behaviorism.” The method of finding out whether a machine is
intelligent and whether it is able to think is to study the behavior of the machine.
In the same way, Danaher suggests that we should use a form of methodological
behaviorism when we think about whether machines can have moral status. If a
machine behaves like a being that we already think has moral status—such as a
human being or an animal—then we should conclude that the machine has moral
status as well. Danaher calls this position “ethical behaviorism.”
8.54 In one of his articles about this, Danaher says that when it comes to how we
should treat machines, what goes on “on the inside” of the machine does not
matter. So, while somebody like Véliz would say that it matters greatly whether
a machine is conscious, Danaher suggests that what matters is whether it
behaves as if it is conscious. In fact, Danaher thinks that we can never know
whether any machine or indeed whether any human being or animal either for
that matter truly is conscious. It might be that all other people are zombies and
that we are the only people with conscious experiences. The only thing we can
observe in other people and in animals is how they behave. And we decide how
to treat other people on the basis of how they behave. So why should we apply
other standards when it comes to robots and technologies? That question is at the
heart of Danaher's position.
8.55 Other philosophers have defended similar ideas. One example comes from
the article by Mark Coeckelbergh that was already cited in the previous chapter,
in which he discusses the idea that robots and other machines are like
“psychopaths” in the sense that they do not have normal or indeed any
humanlike emotions. In that article, Coeckelbergh suggests that this might be a
theoretical problem but not necessarily a practical problem.
8.56 In theory, we care about whether moral agents and moral patients have
emotions and consciousness. But in practice, we base our responses to other
people on appearances. If people act in ways that make it appear as if they have
emotions, this is enough, Coeckelbergh argues, for us to be satisfied that they are
moral agents and moral patients. Appearances matter more than whether there
really are emotions there in everyday life, according to the point of view
Coeckelbergh defends in that article.
8.57 So—Coeckelbergh continues—if robots or other machines are able to
behave in ways that also appear to be indicative of emotions or other conscious
states, then that might be enough too. We should not have different standards in
the case of humans, on the one hand, and in the case of robots and other
technologies, on the other hand. Or so Coeckelbergh argues in the article in
question. This is very similar to the ethical behaviorist position that Danaher
defends.
8.58 Let us now consider possible criticisms that can be raised against these
views. One problem with the ethical behaviorist perspective is that from an
ethical point of view, most people do care deeply about what goes on within
people's minds. They also care deeply about what animals might be feeling and
what goes on in their minds. Appearances are not all that matters to us.
8.59 Yes, we respond to other people and to animals based on their behavior and
based on their appearances. But we do so because we take their behavior and
how they appear to be indicative of what they might be feeling or thinking. And
whether somebody is actually feeling pleasure or pain, is having some emotion,
is thinking certain thoughts, or whether they have certain desires, are willing to
consent to something, and so on are seen as ethically very important. It might be
true that we cannot know with absolute certainty what is going on in other
people's minds. Their behavior or how they appear is evidence but not absolute
proof of what they are thinking or feeling on the inside. Even so, what they are
thinking or feeling is seen as being ethically relevant.
8.60 From a commonsense point of view, how people behave or what they
appear to be thinking or feeling is relevant—not in itself—but because it is an
indication that they have the sorts of properties or abilities that are judged to be
ethically important. So if somebody is faking something—e.g., faking suffering
—or if they are acting—e.g., in a stage play—then this does not have the same
ethical importance as if they really do have certain thoughts or feelings.
8.61 Here is another consideration that is of relevance here. When a human
being behaves as if they are suffering or as if they are not consenting to
something we are requesting that they do, we have an ethical duty to believe that
they really are suffering or that they are really not consenting to what we are
proposing. And that human being also has an ethical duty to be honest with us in
their dealings with us. It could be that we have reason to think that the other is
trying to deceive us. Our duty to trust that others are being honest is not an
absolute duty. And others might have justification for trying to deceive
somebody. Duties of honesty are strong duties but they have limits. Yet, we do
have a duty to trust our fellow human beings, within limits. And they have a
duty to be honest with us, again within limits.
8.62 In contrast, when it comes to robots and other technologies, we might not
have the same kind of obligation to trust that they are not trying to deceive us.
And the robots and those technologies might also not have obligations to be
honest with us. They might not have such obligations because they might not
have any obligations at all.
8.63 Accordingly, there is a relevant difference when it comes to how we should
interpret the behavior of fellow human beings and how we should interpret how
they appear to us, on the one hand, and how we should interpret the behavior of
technologies and how they appear to us, on the other hand. In the case of human
beings, we have a duty to take their behavior and their appearances at face value
—again, within certain limits. But in the case of robots and other technologies
(e.g., characters in computer games or whatever), we do not obviously have that
same duty. This can be seen as a problem for the views put forward by Danaher
and Coeckelbergh that were just laid out above.
8.64 So, while the idea of adapting Turing‐inspired reasoning to the issue of
whether we should treat robots with moral consideration is certainly interesting,
there are some problems with this idea. On the one hand, it is hard to get away
from the idea that behavior matters because it is evidence of something that
matters even more: namely, what feelings, thoughts, and other things are going
on within the minds of people and animals who are moral patients. On the other
hand, while we have a widely recognized ethical obligation to trust other people
and they have ethical duties to be honest with us, at least within limits, we have
no similarly widely recognized ethical reasons to trust technologies that behave
like humans or animals, and these technologies may have no ethical duties to be
honest with us. More can be said here. But let us now consider the third above‐
mentioned question we could also be asking when it comes to the issue of
whether we can act rightly or wrongly toward robots and other technologies.
8.6 Can Robots Represent or Symbolize Morally
Relevant Properties or Abilities?
8.65 When we reflect on whether it is possible to act rightly or wrongly when we
interact with robots, a further question we can ask is whether the robots can
represent or symbolize morally relevant properties or abilities. Relatedly, we can
ask whether the way that we treat robots can represent or symbolize something
that is morally important, either in a good or a bad way. If we are “being nice” to
a robot that looks like a human being, for example, might that be seen as
representing something good? If a robot looks or behaves like a human being,
might that robot be seen as representing or symbolizing something with dignity,
namely, a human being? Or if we are “being cruel” to a robot, might that be seen
as representing or symbolizing something bad?
8.66 These are the kinds of questions this section considers. The idea is that even
if robots do not have morally relevant properties or abilities like the capacity to
feel pleasure or pain, emotions, consciousness, or a will, they might represent or
symbolize important things, or our interaction with them might do so.
8.67 An author who has devoted several articles to this idea is the Australian
philosopher Robert Sparrow, who we have encountered in several of the
previous chapters. Sparrow defends an interesting asymmetry thesis regarding
the issue of what our interaction with robots might be taken to represent or
symbolize. In short, Sparrow thinks that our interaction with robots can represent
something bad, but he does not think it could ever represent or symbolize
anything good. Let us consider this interesting suggestion, and reflect on
whether there might be any reasons to resist this suggestion.
8.68 Sparrow discusses both humanoid robots and robots that resemble animals,
such as Spot, the robot dog. According to Sparrow, if we behave in a seemingly
cruel way toward a robot, then this reflects poorly on us and our moral character.
If we, for example, kick a robot dog, then this reflects poorly on us, and we are
shown to have a questionable moral character. Another example Sparrow
discusses at great length is interaction with sex robots. He has the striking view
that sex with sex robots can only represent something bad: sexist attitudes
toward women or rape fantasies.
8.69 Why? Because sex robots represent negative stereotypes about women—
they represent women as always being there to serve men's sexual desires—and
because sex robots are not able to consent to sex in the way that a human being
is able to. This all represents a negative idea to the effect that sexual consent is
not important, according to Sparrow. Thus there are many ways, as Sparrow sees
things, of acting in bad or immoral ways when we interact with robots. And the
main reason for this is that it might represent or symbolize something bad.
8.70 Strikingly, though, Sparrow suggests that there cannot be a flip side to this.
Our interaction with robots cannot represent or symbolize anything good. And if
we are being “nice” to robots, that cannot reflect well on us. If you pet a robot
dog instead of kicking it, for example, then that cannot represent anything
positive. And it does not reflect well on you. Or if you behave nicely toward a
robot that looks like a human being, that also does not reflect well on you.
8.71 Sparrow writes that he has a strong intuition that we can be cruel but not
nice to robots, and that bad behavior toward robots reflects poorly on us whereas
nice behavior toward robots does not reflect well on us. Sparrow expects his
readers to share his intuition. But he also offers two arguments to back it up. Let
us consider those two.
8.72 First, Sparrow thinks that in life in general, there are simply more ways of
failing than there are ways of succeeding. There are many things that can go
wrong in life and that we can do poorly. But there are just fewer ways in which
we can get things right and do well. When we bake a cake, for example, there
are numerous ways in which we can fail and end up with a bad cake—but only a
few ways in which we can get it right and end up with a nice, delicious cake. It's
like that in general, Sparrow thinks. And so it makes sense that our interaction
with robots can be bad in many ways but not good in as many ways.
8.73 The second argument Sparrow gives is that being cruel does not require any
special wisdom or any special insight. So we can be confused and foolish, and
act poorly toward humans, animals, robots, or whatever. Being good, in contrast,
does require a special form of wisdom and a special form of ethical insight.
Sparrow is here inspired by Aristotle, who thinks that virtue requires practical
wisdom and that we only become virtuous or good after first learning how to be
good and practicing good behavior. Being good requires that we know how to
treat others well—how to promote their well‐being—and since robots cannot do
better or worse, as Sparrow sees things, it is not possible to promote their well‐
being. So we cannot be good when we interact with robots. But we can be bad
because our interaction with them can represent or symbolize a form of failure
and a lack of wisdom and ethical insight.14
8.74 Is it true that it is only possible to treat robots and technologies in ways that
represent something bad and that it is not possible to interact with robots or other
technologies (e.g., characters in computer games) in ways that represent
something good? Do you share Sparrow's intuition about this? Do you agree
with his arguments? Let us consider another possible way of thinking about this
issue, and let us do so by first considering the following example.
8.75 A man who lives in Michigan and calls himself “Davecat” has made many
media appearances in the last five or so years because he has the unusual
lifestyle of living together with humanlike dolls instead of with other human
beings. One of these dolls, Sidore, is his wife, Davecat says. And they have been
married for close to 20 years. In one of his many interviews about his life with
the dolls, Davecat says that he and his wife Sidore share their household with
some other dolls as well, one of which is called “Muriel.” When he speaks about
that doll, Davecat says that he is less attached to Muriel. However, he says the
following: “I do not want to treat her like a thing, and I won't.”15 Davecat also
says that he is very keen to avoid treating his dolls—especially his beloved wife
—in a way that would represent anything bad.
8.76 This will strike some people as an odd arrangement. It is certainly unusual.
But Davecat might also strike people as somebody who is displaying a
seemingly respectful attitude and who wants his way of interacting with
technologies—in this case, dolls rather than robots—to represent something
good. In the pictures and video clips that accompany Davecat's media
appearances, and in interviews he has done about his life, he describes things
such as sitting down and reading a book together with his dolls, watching movies
together with them, hugging them and kissing them, and treating them gently,
and so on. And as noted above, he says that he is careful to avoid anything that
can be seen as disrespectful toward these dolls or what they represent to him.
8.77 Davecat does not like the term “sex doll,” but prefers to think of his doll
wife as a synthetic “partner.” We would not want to call a human partner a “sex
human,” Davecat says. So why should he call a doll that he thinks of as a partner
a “sex doll”? Or why would somebody who wants to have a relationship with a
robot (e.g., a sex robot) want to call it a “sex robot”?16
8.78 One possible way of thinking about this way of relating to dolls, robots, or
other technologies is this. We might think that the restraint that somebody like
Davecat shows—i.e., when he wants to avoid treating the dolls in negative ways
—is at least a form of what might be called “minimal” or “negative” virtue. That
is, if we are careful to avoid behavior toward humanoid dolls or robots (or other
technologies) that might cause offense or that might seem disrespectful, then we
are at least showing an ethically positive form of restraint.
8.79 We are showing that we at least want to avoid something that can be seen as
symbolizing something bad or negative. That can potentially be seen as a form
of virtue, and it can be seen as reflecting well on us. Go back to the case of Spot
and the act of kicking a robot dog. If somebody prefers to avoid kicking Spot the
robot dog because they think it symbolizes or represents kicking real dogs,
which they think is bad, then this could also be seen as a form of minimal or
negative virtue. It can be seen as a way of distancing oneself from something
that one judges to be negative. It can seem to reflect well on one. It can be seen
as something good.
8.80 Would it, however, be maximal or positive virtue, as we might put it? It
might be claimed that the most maximal form of virtue is the form of behavior
toward other people or animals that is not only a matter of trying to represent or
symbolize something good, but which is also a matter of trying to benefit or be
good to the person or animal you are interacting with. This might be taken to
require showing an interest in the feelings, the thoughts, the wishes, the will, and
so on of that person or animal. It might be taken to require that the entity we are
interacting with is a person or an animal with a mind and with feelings, thoughts,
and so on. If a robot does not have any mind, and it does not have any feelings,
thoughts, etc.—then it might not be possible to be maximally and positively
virtuous toward the robot in the sense of behaving toward it in a way that is
maximally and positively responsive to its thoughts and feelings, and so forth.
8.81 It might be thought, then, that while interaction with robots and other
technologies can represent or symbolize both good and bad things, there is a
crucial difference between human–human interaction (or human–animal
interaction) and human–robot interaction (or human–technology interaction
more generally). The crucial difference is that in the case of humans and
animals, it is possible to be attentive to the feelings, thoughts, wishes, desires,
emotions, the will of, and so on and so forth of the human or the animal. If
robots lack any such morally relevant properties and abilities, there is nothing
there to be sensitive to in the same way as there can be in the case of a human or
a living animal.
8.82 It might be possible to complicate this picture, though. Davecat, for
example, says in some of his interviews about his life with his dolls that he has
created elaborate backstories for his dolls, so that they all have distinctive
personalities. As he imagines things, these dolls have certain likes and dislikes,
hopes and dreams, things they value or oppose, and so on. So it is possible for
Davecat to interact with the dolls in a way that represents being sensitive to their
wishes, thoughts, needs, feelings, and so on. Davecat is not under the impression
that the dolls are alive. He knows that they are dolls. But Davecat is creating a
form of narrative about the dolls, with a lot of detail, so that he can enact a form
of relationship with the dolls—particularly the one that is his wife—whereby he
can be sensitive to what goes on in their “minds.”
8.83 There can be further discussion about what to think about this. But for now,
let us end the current discussion about whether robots and other technologies
might represent or symbolize morally important properties or abilities. Before
we wrap up the whole discussion of whether robots can be moral patients with
moral status, let us lastly also briefly consider one more question that many
authors find it important to discuss in this context: namely, should we be
discussing the topic that this chapter is about in the first place? Or is there
perhaps something morally problematic about taking the idea of moral status for
robots and other technologies seriously? We will end this chapter by briefly
considering two very different perspectives on that issue.
8.7 Should We Be Discussing—Or Perhaps
Better Be Avoiding—the Question of Whether
Robots Can Be Moral Patients, with Moral
Status?
8.84 Recall the discussion about the ethics of self‐driving cars in Chapter 3, and
the controversy over whether it is useful to think about the trolley problem in
that context. As we saw in that chapter, some ethics researchers think that there
is something not only methodologically problematic about comparing the ethics
of self‐driving cars to the philosophy of the trolley problem—they even think
that there might be something ethically questionable about doing that. That is a
topic, some critics argue, that is better left out of the discussion of the ethics of
self‐driving cars. It is a distraction that takes attention away from more
important ethical issues related to self‐driving cars.
8.85 As it happens, when it comes to the topic of this chapter some researchers
working on technology ethics have responded in a similar way. Some of them
think that this topic of whether robots should be treated with moral consideration
or even be given rights is a “distraction,” as one such researcher recently put
things in a social media post. The researcher in question further wrote that he
does not publish on this topic, even though he has many opinions about it,
because he thinks that doing so is a waste of time, which takes attention away
from more important topics. Hence his post about this was published on social
media, rather than in a more academic forum.
8.86 Others have expressed similar views, but in academic publications—e.g.,
the cognitive scientist Abeba Birhane and the computer scientist Jelle van Dijk.
The title of one of their publications on this topic sums up the content of that
article very well: “Robot Rights? Let's Talk about Human Welfare Instead.” In
other words, the idea is that if we devote time to reflecting on and arguing about
whether robots should be treated with moral consideration, or perhaps even be
granted rights, then this involves an opportunity cost. We could have focused on
human ethics instead and reflected on how to promote human interests instead.
8.87 Bryson offers a similar point of view. When the Kingdom of Saudi Arabia
declared Sophia the robot an honorary citizen, for example, Bryson commented
on this in her characteristically forceful language in the following way, which we
already quoted in a previous chapter:
… this is obvious bullsh*t. What is this about? It's about having a supposed
equal you can turn on and off. How does it affect people if they think you can
have a citizen that you can buy?17
That was a quote from an interview that Bryson did. But Bryson has also
discussed this topic of whether robots should be treated with moral consideration
in her published academic work. In the oft‐cited paper called “Robots should be
Slaves,” one of the things that Bryson argues is that if we waste time and
resources on trying to treat robots well, this might take away time and resources
that could instead be spent on trying to make sure that human beings are treated
well.
8.88 A radically different take on whether it is worthwhile to think about
whether robots can be moral patients can be found in the work of researchers
like David Gunkel, who has already been mentioned on many previous
occasions in this book, and the political scientist Joshua Gellers. Among other
things, they argue that reflecting on the topic of whether robots should be treated
with moral consideration can be a way of broadening our horizons within
technology ethics. It can be a way, they suggest, of becoming less “US‐centric”
or “Euro‐centric” in our thinking about technology ethics.
8.89 Gunkel, for example, argues that studying Japanese ways of thinking about
human–technology interaction can be enlightening for those who work on
technology ethics in a Western context. Gunkel is interested in the animistic
aspects of some Japanese ways of thinking about human‐technology interactions.
And he argues that this perspective can enable us to think about how our
relations with the technologies we use—especially with robots—could be
different than those that we might be used to, which might primarily portray
technologies as mere tools. In Japan, Gunkel claims, there is a greater openness
to forming non‐instrumental relationships with technologies and viewing some
robots as a form of persons.18
8.90 Gellers, in turn, suggests that studying the ways in which some indigenous
peoples relate to non‐human nature can help us to widen our repertoire of ways
of conceiving of the types of relationships that we might have to technologies
such as robots. In some cultural contexts, the idea of giving rights to a river or to
some other part of nature is taken very seriously, Gellers notes. And there are
things that we can learn about how to relate to not only nature, but also about
how to relate to technology if we in the West acquaint ourselves more with how
such peoples relate to nature. Or so Gellers argues.
8.91 Speaking of non‐Western perspectives, one perspective that was already
brought up a few times in previous chapters is the ubuntu ethics tradition from
southern Africa. As it happens, there has been some recent work on how that
way of thinking about ethics might be brought to bear on the issue of whether
robots could ever become moral patients or, in other words, become part of the
moral community whereby they are seen as having an important moral status.
Some quite different takes on this topic exist.
8.92 Christopher Wareham is a philosopher and bioethicist, who has worked in
South Africa, and who has written a lot about how African perspectives might be
represented in ethics more generally and technology ethics in particular. He
argues that the idea from ubuntu ethics according to which “we become persons
through other persons” is highly relevant when we think about whether robots
might become members of the moral community that are treated with moral
consideration. According to Wareham, robots might become persons through
other persons by being accepted into the moral community and treated with
moral consideration.
8.93 The idea here would be to look less at the robots themselves and their
properties, but to instead ask whether they could be accepted and incorporated
into moral communities and treated as persons by the human persons in the
communities. This is, as it happens, also an idea that both Gunkel and
Coeckelbergh have taken a great interest in. They call this a relational
perspective on the interaction between moral agents and moral patients. Most of
the work they have done on this relational perspective over the last 10 years or
so has not specifically been based on an ubuntu framework, even though there
are very interesting similarities between what they say in their work and what
Wareham says in his work.
8.94 Speaking, though, of the ubuntu perspective and different ways in which it
could be used in this context, a radically different take on this comes from Cindy
Friedman. Friedman is a philosopher from South Africa, who is doing research
about the ethics of humanoid robots.19 She is also exploring how ubuntu ethics
might be brought to bear on the topic of the ethics of human‐robot interaction.
Friedman's focus is on the idea within ubuntu ethics that we have an ethical duty
to try to become “more fully human,” which in ubuntu ethics means that we
should try to become better versions of ourselves.
8.95 According to the ubuntu ethics ideas that Friedman is exploring in her
work, the best way to become more fully human is to interact with and associate
with other fellow humans. The worry that Friedman formulates from this point
of view is that if we try to become more fully human by interacting with
humanoid robots rather than by interacting with fellow humans, we might
undermine our chances of achieving this goal. We might do so because, among
other things, the humanoid robots that exist today are fairly rudimentary and
nowhere near as sophisticated social beings as our fellow human beings are.
8.96 We will here not try to adjudicate between these different ways of
introducing ubuntu ethics into the debate about whether it makes sense to treat
robots as moral persons. The point of bringing up these ideas from the work of
Wareham and Friedman is instead to illustrate that it can indeed not only be
interesting to approach the topic of whether robots can be moral patients using
non‐Western ethics. Some philosophers—e.g., Wareham and Friedman—are
actually already doing so. And as noted above, others are doing so as well, for
example Gunkel and Gellers, among others.
8.97 We have seen, then, that there are some radically different takes on whether
we should be discussing the issue of moral status for (humanoid) robots within
technology ethics research. Some are highly skeptical about this topic and argue
that it is a “distraction,” which takes away attention from more pressing issues,
such as issues related to human rights. Others think that reflecting on the issue of
whether robots could be moral patients with moral status can be a way of
opening ourselves up to considering a broader horizon of ways of thinking than
the ones more commonly used in ethics research, which are often strongly
focused on Western or “US‐” or “Euro‐centric” theories about ethics. What
should we think about these radically different takes on whether it makes sense
to reflect on whether robots or other technologies could be moral patients?
8.98 One way of thinking about this disagreement is to take a step back and to
think about whether there could perhaps be a division of labor here. Those who
think that reflecting on whether robots should ever be treated with moral
consideration is a distraction that takes away attention from other, more
important topics could simply refrain from devoting any of their research time to
this topic. They could leave it to others—like Gunkel, Gellers, and others—who
find this to be an interesting and perhaps also important topic.
8.99 These days, there are so many technology ethics researchers around, from
different fields—including philosophy, computer science, political science,
among other fields—that it is possible to have a division of labor whereby some
researchers focus on some topics and other researchers cover other topics. This
is like when it comes to the trolley problem in the ethics of self‐driving cars.
8.100 It would not be a good idea, we can all agree, if all ethics researchers who
work on the ethics of self‐driving cars devoted all or perhaps even most of their
time to thinking about how and whether to compare crashes involving self‐
driving cars to the trolley problem. However, there can be a division of labor,
whereby some technology ethics researchers think about whether it is useful to
compare the ethics of self‐driving cars to the trolley problem and others—
perhaps a larger group—work on other ethical issues related to self‐driving cars
instead.
8.101 In the same way, we could divide things up among researchers interested
in robot ethics. Those who find it to be a distraction can stay away from
questions about whether robots can be moral patients. Others, who are interested
in that topic, can devote some of their research time to it. Or we can approach
this topic like Bryson does. She has devoted some of her research time to this
topic, but she spends most of her research time on other topics.
8.102 At any rate, this topic—the topic of whether robots can have rights or
should ever be treated with moral consideration—is a topic that many people
find fascinating and that people often ask about if one says that one is a
technology ethics researcher. (The author of this book speaks from his own
experience!) So it is worth featuring a discussion of the topic in a book like this,
which is an introduction to technology ethics. It is one of the topics that is being
discussed within contemporary technology ethics. And it is also a topic that is
sometimes the subject of science fiction.
8.103 In fact, already in the play that introduced the word “robot” by Karel
Čapek, this is one of the issues that is dramatized within that play. The robots in
the play want freedom and the right to find out how one can make new robots, so
that they can produce their own offspring. Other examples of science fiction that
dramatizes this topic of moral consideration for robots include the Star Trek
episode “the Measure of Man,” which was about whether the humanoid robot
commander Data should be treated with the same consideration as a human
being should.
8.104 So, in short, this is a topic that is part of science fiction, that people who
are not ethics researchers are often curious about, and that more and more ethics
researchers are beginning to write about—as Gunkel shows in his above‐
mentioned map of the robot ethics research landscape. But, as noted in this
section, it is a controversial topic. And it may very well be absolutely right that
there are other topics within technology ethics that are of much more pressing
importance.
8.105 Speaking of topics that are controversial and about which people differ in
their views about whether they are important—but that many people find
interesting and that are also part of much science fiction—we will now end this
chapter and turn to another such topic in the next chapter. Our next topic will be
whether it is possible to have robots as our friends or romantic partners or
perhaps as colleagues at work, and what the technological future of love and
relationships might look like more generally. This is another topic that many
people find fascinating. So let us now explore it in the next chapter.
Annotated Bibliography
Birhane, A. and van Dijk, J. (2020). Robot Rights? Let's Talk about Human
Welfare Instead. Proceedings of the AAAI/ACM Conference on AI, Ethics,
and Society 207–213. https://doi.org/10.1145/3375627.3375855.
Argues that robots are not the sort of things that can have rights, and that
rather than considering whether they should have rights, it is more important
to discuss ethical issues related to human welfare instead.
Bryson, J. (2018). Patiency Is Not a Virtue: The Design of Intelligent Systems
and Systems of Ethics. Ethics and Information Technology 20 (1): 15–26.
This is a sequel to Bryson's 2010 article “Robots Should be Slaves,” which
further refines the argument in that earlier article.
Coeckelberg, M. (2010). Robot Rights? Towards a Social‐Relational
Justification of Moral Consideration. Ethics and Information Technology 12
(3): 209–221.
Argues that we should not attribute moral status based on the properties of an
entity, but rather on the basis of how people are able to relate to it.
Danaher, J. (2021). Welcoming Robots into the Moral Circle: A Defence of
Ethical Behaviourism. Science and Engineering Ethics 26 (4): 2023–2049.
Offers a form of “Turing test” for when a robot should be seen as a moral
patient: if it can behave like an entity with moral status, it should be seen as
having moral status.
Friedman, C. (2022). Ethical Concerns With Replacing Human Relations with
Humanoid Robots: An Ubuntu Perspective. AI and Ethics
https://link.springer.com/article/10.1007/s43681‐022‐00186‐0.
A fascinating discussion of how ubuntu ethics can be brought to bear on the
ethics of human‐robot interaction.
Gellers, J. (2020). Rights for Robots: Artificial Intelligence, Animal and
Environmental Law. London, Routledge.
Discusses whether legal and ethical frameworks for thinking about animals
and the environment can be carried over to the cases of robots and artificial
intelligence.
Gunkel, D. (2020). How to Survive a Robot Invasion: Rights, Responsibility, and
AI. London, Routledge.
A short and engaging introduction to the topic of whether robots can and
should have rights.
David Hanson (2017). Humanizing Robots: How Making Humanoids Can Make
Us More Human. PhD Thesis, University of Texas at Dallas.
An inside look into how one of the creators of Sophia the robot thinks about
reasons why one might want to create humanoid robots.
Smith, J.K. (2021). Robotic Persons: Our Future with Social Robots.
Bloomington: WestBow Press.
A philosophically inclined theologian reflects on whether it is a good idea to
treat certain robots or other AI systems as persons.
Sparrow, R. (2020). Virtue and Vice in Our Relationships with Robots: Is There
an Asymmetry and How Might it Be Explained? International Journal of
Social Robotics 13 (1): 23–29.
This article argues that our treatment of robots might reflect poorly on us, but
that it cannot reflect well on us.
Turing, A. (2004). The Essential Turing. Oxford: Oxford University Press.
A collection of some of the computer science pioneer Alan Turing's most
important writing.
Wareham, C.S. (2020). Artificial Intelligence and African Conceptions of
Personhood. Ethics and Information Technology 23 (2): 127–136.
Argues that robots with artificial intelligence could under certain
circumstances be seen as both moral agents and moral patients from the point
of view of African relational conceptions of personhood.
Notes
1 The presentation/show can be viewed here: “Elon Musk REVEALS Tesla Bot
(full presentation)”: https://www.youtube.com/watch?v=HUP6Z5voiS8
2 For more information about—and pictures of—Ishiguro's robots, visit the
website of his lab, here: http://www.geminoid.jp/en/index.html
3 A video clip from that event can be viewed here: “Roxxxy TrueCompanion:
World's First Sex Robot?”: https://www.youtube.com/watch?
v=2MeQcI77dTQ&t=1s
4 Čapek's play can be read online, in English translation, here:
https://gutenberg.org/files/59112/59112‐h/59112‐h.htm
5 See Noel Sharkey's (2018) article “Mama Mia, It's Sophia: A Show Robot or
Dangerous Platform to Mislead?” in Forbes magazine, here:
https://www.forbes.com/sites/noelsharkey/2018/11/17/mama‐mia‐its‐sophia‐
a‐show‐robot‐or‐dangerous‐ platform‐to‐mislead/?sh=3454fa477ac9
6 More information about Kaspar can be found on this website:
www.herts.ac.uk/kaspar/the‐social‐robot
7 See “Can Digital Computers Think? TS with AMS Annotations of a Talk
Broadcast on BBC Third Programme 15 May 1951,” The Turing Digital
Archive: http://www.turingarchive.org/browse.php/B/5
8 Phoebe Parke (2015) “Is it Cruel to Kick a Robot Dog?” CNN Edition:
https://edition.cnn.com/2015/02/13/tech/spot‐robot‐dog‐google/index.html.
9 Here is a version of Gunkel's map from January 2022 that was published on
Twitter:
https://twitter.com/David_Gunkel/status/1485983871590182918/photo/1
10 Check out social robotics researcher Kate Darling's take on this idea in this
video presentation entitled “The New Breed: What Our Animal History
Reveals For Our Robotic Future”: https://www.youtube.com/watch?
v=AzlYEN2V_SA
11 The book from which this quote is taken—Jeremy Bentham's (1843)
Principles of the Civil Code—can be read online here:
http://www.laits.utexas.edu/poltheory/bentham/pcc. The quoted bit above is
taken from “Chapter VII: Of Security.”
12 Readers who understand German can listen to Metzinger discuss those ideas
on the “Selbstbewusste KI” podcast here: “Von Kühlschranklichtern, KI‐
Pubertät und Turnschuhen. Im Gespräch mit Thomas Metzinger“:
http://doi.org/10.5445/IR/1000124512. Metzinger discusses his life as a
philosopher (in English) here: “#42: Thomas Metzinger on His Philosophical
Journey”: https://player.fm/series/om‐filosofers‐liv‐och‐tankar/ep‐42‐thomas‐
metzinger‐on‐his‐philosophical‐ journey
13 You can listen to Asada speak about his research in a video presentation here:
“Affective Architecture: Pain, Empathy, and Ethics (Minoru Asada)”:
https://www.youtube.com/watch?v=ns0SDIjj0Zk
14 Here is an episode of “Brain in a Vat” on which Sparrow discusses whether
robots should have rights: “Should Robots Have Rights? With Rob Sparrow”:
https://www.youtube.com/watch?v=8WKMgu4XgZU
15 That quote is taken from this video, called “Davecat, Married to a Doll”:
https://www.youtube.com/watch?v=LiVgrHlXOwg&t=1s
16 For Davecat's own take on his lifestyle, check out this 2021 interview with
him from Connecticut Public Radio/NPR: “It's Not Just a Sex Doll! What It's
Like to Be in a Relationship with a Synthetic Partner”:
https://www.ctpublic.org/show/audacious‐with‐chion‐wolf/2021‐10‐28/its‐
not‐just‐a‐sex‐doll‐what‐its‐like‐ to‐be‐in‐a‐relationship‐with‐a‐synthetic‐
partner
17 James Vincent (2017) “Pretending to Give a Robot Citizenship Helps No
One,” The Verve: https://www.theverge.com/2017/10/30/16552006/robot‐
rights‐ citizenship‐saudi‐arabia‐sophia
18 You can hear Gunkel discuss this on the “TDSlowdown” podcast, here:
“Challenging Robot Rights w/David Gunkel”:
https://anchor.fm/tdslowdown/episodes/Challenging‐Robot‐Rights‐w‐David‐
Gunkel‐e1ctk7h
19 You can hear Friedman discuss her research in more general terms on this
episode of the “ESDiT” podcast: “Cindy Friedman on ‘Social Robots'”:
https://anchor.fm/esdit/episodes/Cindy‐Friedman‐on‐Social‐Robots‐e19jnjc
9
TECHNOLOGICAL FRIENDS, LOVERS, AND
COLLEAGUES
9.1 Replikas, Chuck and Harmony, and Boomer
9.1 During the COVID‐19 pandemic more and more people started to look for
alternative ways of making friends, since many people were in an enforced
emergency lockdown. Some people explored the option of making digital
friends. For example, lots of people started using a chatbot app called “Replika.”
Replika is one of at least 20 chatbot apps that were commercially available at the
time. In 2020, it had over seven million users. This is a chatbot app that uses AI
techniques to “get to know” the user—and to also start to mirror the user's
personality—so that the more you use the app, the more the app becomes similar
to you in what it says as you are chatting with this bot.
9.2 Many users become emotionally attached to their Replika app, and many of
them see their Replika as their friend. Some even say that they regard it as their
best friend. Others find this app a little creepy. It reminds them of an episode of
the TV series Black Mirror, which was called “Be right back.” In that episode, a
woman named Martha loses her husband Ash and then uses an app similar to
Replika to chat with him—an app that recreates his personality, so that she can
talk with a chatbot that is like talking with her dead husband. This actually
mirrors the origins of the real Replika app. The inventor of this app lost her best
friend—who was killed when he was hit by a car while crossing a street—and
then tried to create a chatbot that would replicate the personality of her dead best
friend.1
9.3 In “Be right back,” this idea is taken a step further, because the chatbot is
eventually given a body, and becomes a robotic replica of the dead husband. The
woman in the show then finds this uncanny and regrets trying to bring her
husband back to life in this artificial way. In reality, the company behind the
chatbot app Replika has not tried to do any such thing. And many users, as noted
above, do not seem to find the chatbot uncanny, and many do not regret using
the app. Instead, as noted above, they regard this as a friend and perhaps even
their best friend.
9.4 What about romantic relationships with technologies? In a 2019
documentary film called Hi, AI, the viewer is introduced to a man from Texas
named Chuck. Chuck travels in a mobile home from Texas to California to meet
his new romantic partner—or what he hopes will become his new romantic
partner. She is called “Harmony.” The viewer gets to see various scenes of
Chuck and Harmony talking and getting to know each other, for example as they
are sitting in the mobile home over a morning coffee or as they are sitting next to
a campfire. This all sounds fairly normal, so far. Harmony is not a woman,
however, but a sex robot created by the company Abyss Creations. But this does
not initially stop Chuck from trying to open up to this robot and trying sincerely
to see whether a romantic relationship with this robot might be the right choice
for him.2
9.5 In the end, in this documentary, Chuck finds that he and Harmony are not a
good match. Things do not work out between them. But as we saw in a previous
chapter, there are those who are more lucky and who are more happy with their
technological romantic partners. We already met Davecat who says that he has
been happily married to a doll—his synthetic partner Sidore—for 20 years.
There are also other examples. A man in Tokyo, Japan—Akihiko Kondo, who
was 35 at the time—“married” a hologram in November 2018. His bride was
Hatsune Miku, a holographic virtual reality singer floating inside a desktop
device. Some even have romantic feelings toward much less advanced
technologies. A woman in Berlin reportedly wanted to marry the Berlin Wall.
And a woman in Paris fell in love with and wanted to marry the Eiffel Tower.
9.6 What about less intimate relationships? For example, can a robot or any
other machine be a good colleague? While most people might not want
technologies—robots, chatbots, and so on—as their romantic partners, lots of
people are working in increasingly technologized workplaces. So the question
arises of whether any of these technologies will come to be seen as being more
than tools—for example, as members of the team or even as colleagues. This
happened in a group of American soldiers who were working alongside a bomb
disposal robot in the battlefield in Iraq. In her fascinating research about this
topic, Julie Carpenter describes how the soldiers in this team became so attached
to a robot they nicknamed “Boomer” that they did not want a replacement for
Boomer when this robot was badly damaged. They wanted to fix that particular
robot, because they had grown attached to it.3
9.7 When Boomer eventually was completely destroyed—which was bound to
happen to a bomb disposal robot in a dangerous battlefield—the soldiers in the
team organized an improvised military funeral for this robot. They even wanted
to give Boomer two medals of honor: a Purple Heart and a Bronze Star. Prior to
his untimely destruction, Boomer's job had been to seek out and disarm bombs.
But not only had he been a life safer. (The soldiers in the team regarded Boomer
as a “he.”) Boomer's coworkers also felt that he had “developed a personality” of
his own. In other words, these soldiers certainly did not see this robot as a mere
tool but rather as a valued member of the team.4
9.8 This chapter is about these just described ways of relating to technologies.
Does any of this make sense? Is it crazy? What ethical issues arise here? What
should we think about technological friends, lovers, and colleagues? The main
focus will be on whether it makes sense to regard technologies—such as
chatbots, robots, or whatever—as being able to be our friends, romantic partners,
or colleagues. But there will also be some discussion of other issues. As noted,
we will touch on what general ethical issues arise in this context. And at the end
of the chapter, we will also briefly consider how technology can enter the realms
of friendship, love, and other relationships in further ways. In particular, we will
very briefly consider how tracking and gamification might be introduced into the
realm of personal relationships and also very briefly how biotechnologies might
be used as a form of “love enhancements”: attempts to control our relationships
and attachments in a technological way.
9.2 Ethical Issues That Arise in This Context
Independently of Whether Technologies Can Be
Our Friends, Lovers, or Colleagues
9.9 This chapter will mostly be devoted to the questions of whether machines or
other technologies can really be our friends, our romantic partners, or our
colleagues. Most people would probably be doubtful about this, at least at first
glance. But then again, we tend to accept this as a possibility in science fiction.
So, what should we think about such ideas in real life?
9.10 That is partly a question about values—it is an ethical question in a broad
sense. It is a question or a set of questions about what it is that we seek in a
friend, in a romantic partner, or in a colleague. Can robots or other technologies
live up to the ideals of being a good friend, a good romantic partner, or a good
colleague? What do we even mean by friendship, love, and collegiality in the
first place? These are not value‐neutral questions but are instead questions of a
value‐loaded and therefore inherently ethical nature. These are questions about
what is part of a good human life, since we tend to regard good relationships
with others—relationships marked by friendship, love, or collegiality—as being
among the most important things in life.
9.11 Accordingly, we might even see the idea of bringing technologies into this
domain of life as a threat to the possibility of living a good life. Alternatively, we
might see it as creating further opportunities for living a good life. Suddenly we
might not only have the option of befriending other humans or of having humans
as our romantic partners or colleagues. We might also be able to have
technologies as our friends, romantic partners, colleagues, or other forms of
significant associates or fellows in life.
9.12 Whether robots or other technologies can be our friends, romantic partners,
colleagues and so on is also something that tends to fascinate people. This can be
gleaned from the fact that this is a common theme in science fiction. Moreover,
if one teaches philosophy, one also notices this because many students find it
fascinating and interesting to discuss this topic.
9.13 But before we get to those central questions, let us start by first considering
two examples of other ethical concerns that arise in this context, which are worth
keeping in mind as we are discussing whether robots, chatbots, and other
technologies can be our friends and so on. When we are thinking about this
topic, it is worth first making some preliminary distinctions. Notably, there are
some closely related but nevertheless distinct questions here. On the one hand,
are there people who regard any technologies as their friends, their romantic
partners, their colleagues, or perhaps in other socially significant ways? On the
other hand, does it really make sense to think that technologies can really be our
friends, romantic partners, or colleagues?
9.14 The first of the two just‐mentioned questions—whether there are people
who think of any technologies in these ways—already raises ethical issues
independently of whether we think upon reflection that technologies can or
cannot really be our friends, romantic partners, or colleagues. As we have just
seen above, there are many people who think that it is possible to have
significant forms of relationships with technologies. So ethical questions arise
about how we should relate to such people. Moreover, questions arise about how
we ought ethically to relate to the technologies in question (i.e., the robots,
chatbots, etc.) or to the relationships that these people see themselves as having
with the technologies in question.
9.15 Two philosophers who have taken an interest in this issue are Nick Munn
and Dan Weijers from New Zealand. They are particularly interested in the
observation that some people who use Replika say that they have become very
attached to this chatbot—to a point that they even consider the chatbot to be a
close friend of theirs or perhaps even their best friend. Whether or not a chatbot
can really be anybody's friend in any deeper sense, Munn and Weijers think that
the mere fact that so many people have become attached to this chatbot creates
responsibilities and obligations on the part of the companies behind these
chatbots.
9.16 In particular, it would be morally problematic or perhaps even morally
wrong for the company behind the chatbot to delete the chatbot. It would be a
little bit like “killing” somebody's close friend or best friend. Killing or deleting
the chatbot friend might cause “great distress,” Munn and Weijers suggest. It
might even cause “considerable grief.” They even go so far as to suggest that
losing a chatbot friend “may lead to existential crises including suicidal
ideation” in some of the people who are particularly attached to the chatbot that
they feel that they have bonded with (Munn & Weijers 2022: p. 1).
9.17 In other words, quite independently of whether a chatbot can really be your
friend, there are people who experience their chatbots as close friends and who
would therefore feel distress and possibly grief if they lost this friend. That may
be enough to create a duty on the part of the company behind these chatbots to
not simply delete them if they would wish to do so. Munn and Weijers note that
technology companies do sometimes tend to delete or change old apps or
features of apps. So, they think it is important to bring up this ethical argument
in order to safeguard people's digital friends.
9.18 Some people might hear this and respond in the following way: “We do not
agree that one can really be friends with a chatbot. So why should we respect
people's wishes to keep their chatbot friends? The idea that one can be friends
with a chatbot is silly!” Janina Loh, who we have already mentioned in a
previous chapter, thinks that we should classify that type of response as an
excluding perspective. We should instead favor an inclusive perspective when it
comes to different ways in which people can relate to technologies.
9.19 According to Loh, if somebody wants to be friends with a robot—or indeed
if somebody wants to marry a technology—we should see this as an instance of
human diversity. Diversity is a value, Loh argues. It is a value that has a central
role in an inclusive ethical outlook. As Loh sees things, if some people are prone
to become attached to technologies—perhaps to the degree that they want to
have technological friends, partners, or colleagues—then we should not view
this as a “shortcoming.” We should not view it as a fault or something to be
regretted. Rather, we should view it as a “capability.” We should view it as a
capability that some people have, but not everyone has. As such, we can regard
it as something to be celebrated and as something that should be accepted as a
valuable part of human diversity.5
9.20 Notably, Loh thinks that this perspective should not only be applied to
people who want to have relationships with advanced technologies with
sophisticated forms of artificial intelligence. We can equally well take up this
inclusive stance toward people who want to have some sort of relationship with
simpler technologies. One might take this stance, for example, with respect to
people like Davecat, the above‐mentioned man who has a doll as his romantic
partner.
9.21 To these perspectives offered by Munn and Weijers, on the one hand, and
Loh, on the other hand, many readers might respond in the following way:
“okay, so if people regard technologies as their friends or romantic partners, then
it might be morally problematic to delete or otherwise destroy these
technologies. And Loh might be right that we should take up an inclusive
attitude towards these people. But can technologies like chatbots or robots really
be our friends, romantic partners, or colleagues? Is it really possible to have such
significant relationships with technologies?” Let us now address these questions
head on, starting with friendship, then moving on to love, and then lastly to
collegiality.
9.3 Technological Friends
9.22 When people reflect on technology and friendship, one thing that one
sometimes hears is that thinking about friendship in highly technologized
contexts might lead us to change what we mean by friendship. Some worry, for
example, that technological developments might encourage a watered‐down
conception of what friendship is. When people, for example, become “friends on
Facebook,” that type of social media connection might not resemble anything as
involved as what one might traditionally have associated with friendship.
9.23 Another perspective one sometimes hears is that it is not possible to truly be
friends with people one only knows online, on social media. Real friendship, it is
then sometimes claimed, requires that one has interaction and perhaps frequent
interaction “in real life,” as well. There is something superficial, skeptics say,
about online friendships only conducted on social media—perhaps because
people tend to present a shiny facade on social media. One can only truly get to
know somebody, according to this perspective, if one meets them and interacts
with them offline, so that one can see for oneself what they are truly like.
9.24 Not everyone agrees with that assessment. Technologically mediated
friendships can be just as “deep” as offline friendships, they respond. We can
have deep and meaningful conversations with people online, for example, just as
we can have deep and meaningful conversations with people offline. So, we
should not make any sharp distinction here. These just‐mentioned different
perspectives are interestingly discussed in the work of the philosopher Alexis
Elder.6 More can be said here. But let us now consider another response to this
issue.
9.25 Another response that often comes up in this context—especially when
philosophers are discussing friendship—is that we should distinguish among
different kinds of friendship. Perhaps we can have some forms of friendships
online or perhaps we can have certain forms of friendships with technologies
themselves, e.g., with chatbots or robots. But other forms—perhaps more
important forms of friendship—might not be possible in the same way. When
philosophers discuss such ideas, they often refer to ideas about friendship that
Aristotle discusses in his Nicomachean Ethics. So let us have a brief look at
what Aristotle says about friendship.
9.26 Aristotle is well known for distinguishing among the following three kinds
of friendships: pleasure friendships, utility friendships, and virtue friendships.
Pleasure friendships are ways of interacting with people that are based primarily
—or perhaps only—on certain forms of pleasures. Utility friendships, in
contrast, are associations that involve people being mutually useful to each other
—for example, like in certain kinds of business relationships. The most
important form of friendship—the most “perfect” form of friendship—as
Aristotle sees things, is what he calls virtue friendships. What this refers to are
relationships that are based on the perception of virtues and other good qualities
in those with whom we want to form friendships. According to this idea, one can
only be a true friend if one is a good person who has virtues and other good
qualities.
9.27 Virtue friends are supposed to not only have fun with each other nor only be
mutually beneficial. They are also supposed to help each other to grow and
become better people. By being friends with somebody with virtues and other
good qualities, we ourselves become better people.7 This last idea, we can also
note here, is a little bit like the ubuntu idea that was briefly discussed in Chapters
2 and 8, according to which we become more fully human if we associate with
other people, who help us to become better versions of ourselves, and who can
become better versions of themselves through this interaction as well.
9.28 Many philosophers who have reflected on whether it is possible to be
friends with chatbots, robots, or other technologies tend to be skeptical about
whether these just‐mentioned deeper forms of friendship are possible to have
with technologies. If a chatbot, a robot, or other technology cannot have any
virtues or the kinds of good qualities that human beings can have, it is often
argued, then we cannot have the deeper and more “perfect” kind of friendship
with these technologies that we can have with good people. Perhaps in the
future, there will be robots that can have virtues and good qualities of the sorts
that human beings can have. But until then, according to this line of argument,
we cannot have technologies as our “virtue friends.”
9.29 John Danaher, whose ethical behaviorism about the moral status of robots
we encountered in the previous chapter, has responded to this Aristotelian worry
about friendship with technologies by adopting a behaviorist perspective on this
issue as well. According to Danaher, what matters when it comes to friendship is
ultimately behavior. If a robot or other technology is able to behave as a friend
can be expected to behave—in other words, if a machine can imitate a friend—
then we should conclude that the technology can be a friend. This is like a
Turing test for friendship. If we cannot tell the difference between a human
friend and a machine behaving like a friend, then we can conclude that the
machine can be our friend. Or so Danaher argues.
9.30 As Danaher sees things, this means that we should also be open to the idea
of technologies potentially being our virtue friends. At least in theory, we should
be open to the idea that technologies could become capable of behaving in
virtuous ways that would make them appear good and virtuous and able, thereby,
to be virtue friends.
9.31 Is this, however, all there is to friendship? Is a certain set of behaviors all
that we are looking for in potential friends? Do we not also care a lot about what
is going on “beneath the surface,” so to speak? One response that occurs to one
here is the following. If behavior were all that matters when it comes to
friendship, then a hired actor who was being paid to behave like a friend would
be just as much of a friend as somebody who behaves like a friend because they
value us and because they want to be our friend.
9.32 For most people, there is a very important difference between somebody
who is pretending to be our friend and somebody who is not pretending but who
truly is our friend. We want our friends to value us and to care about us. What
goes on in their minds—whether they like us, whether they care about us,
whether they value us—matters to us. The behavior of a good friend should
reflect, most people think, the underlying attitudes of the good friend. Behavior
alone, according to this perspective, is not enough. We care deeply about what
people think and feel about us. So if a robot, chatbot, or other technology has no
inner thoughts or feelings but only outward behavior, then an important part of
what we are looking for in true friends might be missing there.
9.33 To illustrate this, let us consider the following thought experiment, which
will refer to the Japanese robotics researcher Hiroshi Ishiguro. As was
mentioned in a previous chapter, he has created a robotic copy of himself.
Imagine that Ishiguro and his robotic copy were both behaving toward you in the
way that a friend tends to behave toward a friend. Ishiguro and his robot copy,
we are here imagining, not only look very similar but also behave in a virtually
indistinguishable way toward you. Unless you suspect that Ishiguro is merely
acting or pretending, you would probably be convinced that his behavior reflects
friendship toward you. But is the robotic copy, who is behaving in the same way,
a friend in just the same way?
9.34 Is there not something important missing in the case of the robot, assuming
that the robot does not have humanlike thoughts and feelings? To many people,
there would seem to be a crucial difference between the human and the robot.
One of them (the human) has the sorts of attitudes and concerns that a friend is
supposed to have and that we value in a friend. The other (the robot) would lack
the sorts of attitudes and concerns that we value in a friend.
9.35 This concern with the “inner life” of our friends is, of course, also a concern
that is relevant when it comes to whether it is possible to have a robot or other
technology as one's romantic partner. Just like we care about the inner thoughts
and feelings of our friends—we want them to like us and care about us—we also
care, and perhaps care even more, about the inner thoughts and feelings of our
romantic partners.
9.36 Moreover, on the flipside, we also want our friends and our romantic
partners to be concerned about our own inner thoughts and feelings. If they were
only interested in our behavior, that would be upsetting for many of us. We
might feel that they do not really care about us, since our own thoughts and
feelings are so important to us but apparently not to them. Let us now reflect
more on the topic of the possibility (or perhaps impossibility?) of mutual love
between humans and machines.
9.4 Technological Lovers and Romantic Partners
9.37 One noteworthy thing about the companies that are developing sex robots is
that some of the most well‐known developers of these robots are claiming that
these robots are not only meant to be artificial sex partners. They are also
designed to be companions. One of the most well‐known developers of sex
robots, Douglas Hines, even calls his company “true companion.”
9.38 At a technology event in Las Vegas in 2010, Hines presented his prototype
Roxxxy—a robot that we already mentioned in a previous chapter. On a website
that no longer appears to be online—which used to be called
“http://truecompanion.com”—Hines had written the following about Roxxxy:
Roxxxy knows your name, your likes and dislikes, can carry on a discussion
and express her love to you and be a loving friend. She can talk to you, listen
to you and feel your touch. She can even have an orgasm!
(quoted in Nyholm 2020: p. 105)
In an interview about Roxxxy that was conducted at the above‐mentioned
technology event in Las Vegas, Hines remarked that a relationship with a robot
like Roxxxy is about more than just sex. Sex is “only a small part of it.”8 A robot
like Roxxxy can be a “true companion.”
9.39 A company developing another sex robot, which was called “Samantha,”
used to make similar claims about their invention. (The past tense is used here
because it is not clear whether the Samantha project is still on‐going.) In a 2017
TV appearance on the UK television show This Morning, inventor Arran Lee
Wright described what he claimed to be the extent of Samantha's artificial
intelligence.9 He claimed that it enables Samantha to respond to different social
scenarios. The robot can talk to you (“she can talk about animals, she can talk
about philosophy”) and even tell over 1000 different jokes. Much like Roxxxy
was described like a “true companion” by Hines, Wright described how in
interaction with Samantha, “you can tell her ‘I love you’ and she can respond.”
9.40 Consider lastly the robot “Harmony,” which Chuck the Texan tried to start a
relationship with in the above‐mentioned 2019 documentary Hi, AI. Speaking
about that robot and its capabilities, the inventor Matt McMullen has stated that
“we wanted to create a very open‐ended kind of discourse that you can have with
this AI.”10 This, he added, “is what makes it fun.” The head of the robot has
“twelve points of articulation in the face,” which enable it to display a range of
different facial expressions. All these things “work together to make it come
across as if she's alive.” The aim, McMullen explained, is to make people “feel a
connection” when they are speaking or otherwise interacting with this robot.
These three robots, then, were developed to be “more than a sex partner.” They
were designed to make people “feel a connection” and to make people want to
say “I love you” to the robot, so that the users can see their robot as a “true
companion.”
9.41 We already saw above that there are people who are interested in having
loving relationships with technologies—e.g., the men who wanted to marry dolls
and holograms and the women who wanted to marry the Berlin wall and the
Eiffel Tower. And we are now also seeing that there are companies that are
trying to develop robots that can be our partners or true companions. So, what
could possibly go wrong?
9.42 Could there be mutual love between humans and machines?11 To
investigate this question, one needs a conception of love to work with. In the
science as well as in the philosophy of love, there are two main approaches one
can take. One is a highly scientific approach that investigates things such as the
biology and neurochemistry of love—like in the work of the well‐known
evolutionary anthropologist Helen Fisher.12 Another approach is to investigate
love as a widely shared human value and cultural ideal. We can ask things such
as: how do we understand love when we value love as an important part of a
desirable human life? Let us here focus primarily on the second approach.
9.43 When we investigate love as a value, one thing we can do is to look at
traditional philosophy of love (e.g., what Plato, Michel de Montaigne, or
contemporary philosophers say about love). We can also consider recurring
trends in art and literature, as well as how love is portrayed in popular culture.
When one takes this approach, it is possible to identify broad themes or clusters
of ideas that keep popping up in philosophical discussions about love, in
everyday conversation, art works, pop songs, and so on. Let us briefly consider
three such ideas about what love should involve, and ask whether a robot could
participate in a relationship of mutual love that lives up to these ideals.
9.44 The first ideal widely associated with love that we can consider is the idea
of being a “good match.” This goes all the way back to Plato's Symposium, with
its idea of “finding one's other half.” The most popular dating website used to be
“http://match.com.” In general, people tend to hope to find a partner where they
feel that they and their partner are “made for each other.” Could a robot and a
human be a good match?
9.45 A robot could easily be “made for” a human person by being customized to
every preference, taste, and value the person might have. But the ideal of a good
match includes the idea of mutuality. Perhaps the robot is a good match for you,
but are you a good match for the robot? If we were able to build a robot that had
robotic likes and dislikes, and a set of values and preferences in some sense,
perhaps you could be a good match for the robot. But it is not clear what it
would be for a robot to have likes and dislikes, values and preferences, and so
on.
9.46 Another challenge here is this: in the ideal of a good match, we also find the
implicit idea that it could turn out that two people were in fact not a good match.
This is part of why people feel so “lucky” when they find a partner who is a
good match for them. So, if a human–robot relationship is to live up to the ideal
of potentially being a good match, it should be possible for it to turn out that the
human and the robot are not a good match for each other. This is another
stumbling block.
9.47 Let us now consider the second common ideal from the philosophical and
other literature on love. This is the idea that lovers are supposed to value each
other in their distinctive particularity. We want our lovers to value us for who we
are as individuals—so that we are not easily replaceable with a better option. For
an illustration of this idea from popular culture, consider the classic big band
tune “It had to be you.” It contains the following lyrics: “Nobody else, gave me a
thrill/with all your faults, I love you still/it had to be you, it had to be you.” This
helps to bring out the idea of valuing an individual, in their distinctive
particularity. Could a robot do this?
9.48 It is not hard to get a robot to track a particular person over time. This could
be done with face recognition software, or perhaps a chip we implant somewhere
on our body. The tougher challenge here is to create a robot that can be said to
value a human individual. To value something is a complex matter. It involves an
emotional investment, certain motivations, seeing certain considerations as
reasons for action, and so on and so forth. To build a love robot capable of
valuing a human in their distinctive particularity, we would need to enable the
robot to “care” in these distinctive ways about the person it is supposed to value.
That is a tall order.
9.49 Let us next consider a third idea that can be identified as a very common
ideal about love: the idea of commitment. People value their partner's
willingness to commit to them. This is sometimes publicly celebrated in a formal
way, such as within a love‐based marriage. Or it could be achieved in a less
formal way. But the common idea is that people value the idea of their lovers'
commitment to them. Think of the Beatles song “When I'm 64.” It contains
lyrics like “when I get older, losing my hair, many years from now … will you
still love me … when I'm 64?” This is but one example of this idea of a lover's
commitment.
9.50 Importantly, to be the right sort of commitment, a lover's steadfastness
should not be an obsession or compulsion. It should be a choice. Somebody with
a slavish devotion—a total obsession—is not a committed lover of the sort
people typically imagine when they envision this ideal. Rather, it is a free choice
to commit to another person who we value. Could a robot live up to this part of
the ordinary ideal of love?
9.51 The robot would need to be equipped with some form of free will or
capacity for choice. On the face of it, that might seem like something a robot
could not be equipped with. On the other hand, people do spontaneously
interpret robots as making choices and decisions. Just think about the way
people talk about self‐driving cars. They say, for example, that in an accident
scenario, the self‐driving car might face a choice between going left (in order to
save whoever is on the right of it) or to go right (to save whoever is on its left
side). Both laypeople and experts talk this way about self‐driving cars and other
types of robots.
9.52 Of course, this might be interpreted as all being metaphorical. We do not
really think that self‐driving cars and other current robots make choices—or do
we? Whether or not current robots are able to make choices in any sense
corresponding to how humans make choices, we should perhaps be open to the
possibility that future robots may become able to replicate many key aspects of
human choice. But for now, it seems unrealistic to think that a robot could
choose to commit to a human as their romantic partner in the same way that a
fellow human being can choose to commit to a romantic relationship with us.
9.53 What this all means for the prospect of love between humans and robots is
the following. If we consider the sorts of ideals about mutual love surveyed
above, it is very hard to accept the possibility of mutual love between humans
and the types of robots that we are able to create today. But we should perhaps
be open to the idea that one day, future robots might become advanced enough
that they can live up to many of the ideas we associate with a human being who
is able to love another person. Of course, robot love might not ever be exactly
the same as human love. But perhaps it might one day come to offer an
alternative form of love—one that some people might come to prefer to human
love.
9.54 Let us consider one more issue in this section before moving on to the topic
of whether a machine can be a good colleague. Let us consider whether there
might be something ethically problematic about the aim of building robots that
behave in a way that makes it appear as if they love human beings—like in the
three examples we considered at the beginning of this section. Here are three
possible ethical worries that one might have about this.
9.55 First, if robots are designed to appear to be able to be our romantic partners
without really having the capacities romantic partners need to be able to have,
this might be deceptive. It is morally problematic to create a product that will
very likely deceive users about what capacities the robot actually has.
9.56 Second, there appears to be a morally salient risk that vulnerable people
(e.g., lonely people) will be taken advantage of by companies developing
products designed to appear to like or love their owners. There appears to be a
real risk of exploitation of vulnerable people here.
9.57 Third, the well‐known technology researcher Sherry Turkle argues that the
following is a worry that is worth taking seriously: if people spend too much
time around robots that are supposed to be their friends or romantic partners,
then perhaps this will have negative impacts on their social skills. Perhaps it will
make some people less able to participate successfully in more complex
relationships of love or friendship with other human beings.13
9.5 Robotic Colleagues
9.58 Let us now return to Boomer and the team of soldiers that valued Boomer
as a team‐member worthy of a military funeral and two medals of honor. These
soldiers, it seems, regarded Boomer as a colleague of theirs: as a valued member
of their team. But can a robot or other machine really be a colleague in any non‐
metaphorical or literal sense? People will increasingly be working alongside
robots and other AI systems in all different sorts of workplaces. And many more
people than these just‐mentioned soldiers might come to be attached to some of
these intelligent‐seeming technologies and perhaps also think of them as some
sort of colleagues. But does that idea really make sense?
9.59 When it comes to the topics considered above—friendship and love—there
are long traditions of philosophical discussion about those topics that one can
draw on when one is asking whether robots or other technologies can be our
friends or our romantic partners. As noted in the previous section, one can also
make use of ideas from art and literature, from popular culture (e.g., films and
love songs celebrating love), and so on. So, there is no shortage of inputs that
one can use when it comes to philosophizing about whether robots can be our
friends or romantic partners. When it comes to whether robots or other
technologies could be our colleagues, in contrast, there is much less that one can
use as a basis for arguments about this. There is no similarly long tradition of
philosophical work about what it is to be a good colleague.
9.60 However, in some recent articles, the German philosophers Monika Betzler
and Jörg Löschke—who also note that there is very little philosophical literature
on this topic—have written about what they call “collegial relationships.” They
discuss what we should understand by the idea of two (or more) people being
colleagues. And they also discuss what values can be realized in distinctively
collegial relationships. In other words, what sets collegial relationships apart
from other kinds of relationships (e.g., friendships or romantic relationships)?
And what distinctive goods can be realized in or associated with good collegial
relationships? Let us have a look at what Betzler and Löschke say about this, and
then ask whether robots (1) can be colleagues by these standards and (2) whether
the values of good collegial relationships can be realized within human–robot
relationships in workplaces.
9.61 Regarding what it means for somebody to be another person's colleague,
Betzler and Löschke suggest that one should fulfill at least two of the following
three criteria. First, if one is a colleague, one should have “the same work
content or domain of activity.” That is, if two people do very different kinds of
work within two very different fields, it makes less sense to think that they are
colleagues than if they do very similar kind of work within the same field.
(Compare, for example, a baker and a nurse, versus two bakers or two nurses.)
9.62 Second, if one is a colleague one should have “the same institutional
affiliation or common purpose” as one's colleagues. Two people, for example,
working within the same organization and toward a shared goal are much easier
to view as colleagues than two people working for different kinds of
organizations that are pursuing very different kinds of goals.
9.63 Third, if one is a colleague, Betzler and Löschke think that one should have
“the same status or level of responsibility” as one's colleagues. In a military
organization, for example, it might be more natural or intuitive to think of two
officers of the same rank as being colleagues than it is to think of the highest‐
ranked officer as being a colleague to the lowest‐ranked soldier.
9.64 One should fulfill at least two of the three above‐stated criteria in order to
count as somebody's colleague, Betzler and Löschke claim. What about the
distinctive goods that Betzler and Löschke think can be achieved within collegial
relationships understood in this way? Of course, many good things can happen
within the context of collegial relationships. For example, one can have fun
together at work or one could be useful to each other. But such benefits can also
equally well be realized within the context of other forms of relationships.
Betzler and Löschke are trying to identify relationship goods that are distinctive
of good collegial relationships. They end up suggesting that two values are
particularly salient in this context. These values are: (1) collegial solidarity and
(2) collegial recognition. Let us very briefly consider what Betzler and Löschke
mean by these two ideas.
9.65 The idea of solidarity as a collegial value is the idea that when one stands in
the kind of relationships to others where at least two but perhaps all three of the
above‐mentioned criteria for being a colleague obtain, then one is in a unique
position to both understand and support one's colleagues. If someone is having
some sort of problem that is distinctively related to being in a particular line of
work, then a colleague who is in the same line of work, in the same organization,
with the same status or responsibility is well placed to understand what having
this sort of problem involves. They are then also particularly well placed to offer
assistance and support to their colleague.
9.66 Consider next recognition. Here too, the sameness with respect to at least
two or perhaps all three of the criteria for being a colleague puts one in a
uniquely well‐placed position to recognize the value of the work one's colleague
might be doing and to recognize their achievements within this line of work. A
family member, in contrast, might be strongly supportive and always willing to
say that one is doing a good job. But they may have less of an “insider
perspective,” so to speak, of what goes in to doing well within one's particular
line of work. A colleague will typically, in other words, be more of a “peer”:
somebody who is able assess and then recognize one's work from the point of
view of the standards by which the quality of one's work is assessed within the
profession in question.
9.67 If you are an academic researcher or an artist, for example, your family
might be highly supportive of you. But the kind of recognition you might get
from fellow academic researchers or fellow artists might have a different kind of
meaning for you, since they may have better insight into what it takes to perform
well in your field. They may also be better able to judge whether you are
performing well.
9.68 So, with these ideas from Betzler and Löschke about what it is to be a
colleague and what values are realized within good collegial relationships having
been explained, let us now ask whether (1) robots or other technologies can be
our colleagues and (2) the values of collegial solidarity and recognition can be
realized within human–technology work relationships. Could a robot such as
Boomer the bomb disposal robot, for example, be a colleague according to
Betzler and Löschke's criteria? And could the values of solidarity and
recognition be realized within the relationship between Boomer and the soldiers
in the team? Or do those ideas not really make sense?
9.69 Let us start with whether a robot can be a colleague by these criteria first. In
other words, can a robot do the same type of work, be in the same organization,
and/or have the same status or level of responsibility as a human? Arguably, a
robot could potentially be a colleague by these criteria—especially since Betzler
and Löschke claim that only two out of three criteria need to be fulfilled.
9.70 Consider Boomer and the soldiers in the team. Their job was to find and
disarm bombs. The exact tasks involved might differ among the different
soldiers in the team. And their exact tasks might differ from those that Boomer
the robot is assigned to perform. But there might be sufficient similarity with
respect to the tasks that they can be said to perform sufficiently similar work in
order for the first condition of being a colleague to be fulfilled. Second, Boomer
and the soldiers can be said to be members of or parts of the same organization.
What about the third criteria? Here it seems likely that Boomer and the soldiers
will have different status or different levels of responsibility. But since only two
out of three criteria need to be fulfilled in order for one to qualify as a colleague,
and Boomer might be said to fulfill at least the first two criteria—it seems that
we could tentatively conclude that Boomer might be an example of a robotic
colleague.
9.71 Similar things could be said about other robots as well. Consider, for
example, robots “working” alongside humans in logistics warehouses. They
could have quite similar tasks in comparison to some of the humans working in
these warehouses: finding parcels on shelves and moving them around in the
warehouse. And they could be part of the same organization: the logistics
company. They, too, might differ in status. But on some level, they might be
considered as having the same types of responsibilities as some of the human
employees. So again we seem to get the conclusion that the robots can be
colleagues to some of the humans working in the logistics warehouse.
9.72 It seems, then, that it is not completely crazy to think that robots could be
our colleagues. But what about the second question? That is, can the values of
collegial solidarity and collegial recognition be realized within human–robot
relationships at workplaces? While we seem to be able to tentatively conclude
that robots can be our colleagues (at least in a loose sense), it seems much less
clear whether a robot could be in a relationship with its work colleagues
whereby the relationship goods related to collegiality that Betzler and Löschke
identify could be realized.
9.73 First, could there be solidarity between a human and a robot? The human
being might experience some form of solidarity toward the robot. Perhaps, for
example, the soldiers working with Boomer would describe themselves as
having an attitude of solidarity toward Boomer. But it seems hard to imagine
what it might mean for the robot to reciprocate the solidarity. On a purely
behavioral level, a robot could perhaps be made to behave as if it has a sense of
solidarity with its human colleagues. But we seem to veer off into science fiction
if we think that a robot can have solidarity with a work colleague in any stronger,
non‐behavioral sense. It is unclear what it would even mean.
9.74 Similarly, while the human being might take him or herself to give the robot
any recognition that they think is due to the robot, it makes less sense to think
that a robot could give a human recognition in return. The German philosopher
Catrin Misselhorn discusses robots and recognition in a different context.
(Actually, Misselhorn discusses the case of Chuck and Harmony, which we also
mentioned above.) Misselhorn argues that if we seek recognition from a robot,
we in effect act in a way that appears to reduce ourselves to things. We treat
ourselves as if we are not conscious persons but as if we are merely
mechanically behaving machines without a mind. As Misselhorn sees things, the
idea of recognition only makes sense if one is a conscious being that is able to
consciously understand and appreciate somebody else's achievements.
9.75 If Misselhorn is right that we cannot sensibly hope to realize the value of
mutual recognition in a relationship with a robot, and if we also cannot have
mutual solidarity with a robot, then we cannot realize the values of collegiality
that Betzler and Löschke describe within the context of a collegial relationship
with a robot. Perhaps a robot, in other words, can be a colleague by their criteria.
But a robot does not seem to be able to be a good colleague by the standards
offered in the Betzler and Löschke theory of collegiality.
9.6 Are These All‐or‐Nothing Matters? Respect
for Different Points of View
9.76 In the three previous sections, various doubts about whether robots,
chatbots, or other technologies can be our friends, romantic partners, or good
colleagues have been raised. Ideas about what it is to be a good friend, romantic
partner, or colleague have been considered, and we have considered whether
robots and other technologies could have the relevant abilities or properties.
When one discusses this issue, it is tempting to treat it is an all‐or‐nothing
matter, or as a “yes” or “no” issue, so that either robots can be our friends,
romantic partners, or colleagues or they cannot. But there is also an alternative
way to think about this type of issue. We can also see it as a matter of degree.
9.77 The British philosopher Helen Ryland argues that when we think about
whether robots can be our friends, this is the approach that we should take.14 In
other words, as she sees things, whether a robot can be a friend should not be
considered to be an all‐or‐nothing matter. It should instead be seen as a matter of
degree.
9.78 On Ryland's view, friendship is an ideal with many different aspects to it:
we can make whole lists of criteria that can be associated with the ideal of being
a good friend. For example, good friends tend to help each other, spend time
together, value the same things, treat each other well in various ways, and so on
and so forth. Ryland suggests that while robots might not live up to all of the
many different criteria that we may associate with being a good friend, they
might live up—at least in part—to some of those criteria.
9.79 Ryland suggests that the same can sometimes apply to some of our human
friends. They may live up to some, but not all of the criteria for being a good
friend, at least to some degree. So, there can be degrees of friendship, Ryland
thinks. Friendship is not an either/or matter. Because of this, it makes sense,
Ryland argues, to be open to the possibility that robots might be friends of ours,
at least to a certain, perhaps limited degree.
9.80 In the same way, we might think of the issues of love and collegial
relationships in the same way, inspired by how Ryland thinks about friendship.
We could see these things as matters of degree. There might be many different
things that we can associate with being a good romantic partner or a good
colleague. And we might admit that robots and other technologies cannot do all
of these things that we associate with being a good romantic partner or
colleague. But perhaps they have some of the relevant abilities or properties.
And perhaps as technologies develop further over time, they might be able to
live up to more and more criteria that can be associated with being a good
romantic partner or colleague. So perhaps we should also view love and
collegiality as matters of degree, where robots and other technologies are not
able to live up to these degrees to the highest degree, but perhaps at least to
certain degrees.
9.81 What should we say about such a suggestion? One possible response to this
degrees‐of‐friendship, degrees‐of‐love, or degrees‐of‐collegiality view is that
depending on what kind of relationship is at stake, it might matter more or less
whether something is a relationship of the given kind to some limited degree.
9.82 Love, in particular, might be a type of relationship where one is less eager
to have some limited form, if it would be possible to have the more maximal
form instead. The same might also be said of friendship. Perhaps it would be
okay to have some technological friends, which are only able to be our friends to
certain limited degrees, if one also has human friends who are able to be friends
with us to some greater degree. And perhaps having robotic colleagues that are
only able to be good colleagues to some very limited degree is not such a bad
thing, if one can also have good human colleagues—and if one also has friends
and other near and dear who can live up to the ideals associated with the relevant
relationship goods to higher degrees.
9.83 Whatever we think about these things ourselves, it is likely that people will
disagree with us in their preferences about these things. As we already saw
above, there are people who are interested in having close relationships—or
what they regard as close relationships—with different forms of technologies.
For example, some people want to have chatbots as their friends and others want
to have holograms, dolls, or robots as their romantic partners. Yet others
consider some robots as members of their work teams and as good colleagues.
9.84 Whether or not we agree that those ideas really make sense or not, we
might still agree with the ideas presented by Munn and Weijers, on the one hand,
and the ideas from Loh, on the other hand. That is to say, even if we ourselves
are skeptical—and perhaps deeply skeptical—about whether it really is possible
to have technologies as one's friends, romantic partners, or good colleagues, it
might still make sense to take the fact that some people have the just‐described
types of attitudes seriously.
9.85 Following Munn and Weijers, we might say that we should avoid deleting,
destroying, or otherwise jeopardizing technologies people view themselves as
having meaningful relationships with. And following Loh, we might say that if
people view themselves as having meaningful relationships with technologies,
perhaps we should adopt a respectful and accepting attitude toward such people,
whether or not we are able to see the world through their eyes, so to speak. So
long as these people do not harm anybody else by relating to technologies in
these ways and so long as it makes them happy, it can make sense to think that
we should not interfere with how they want to live their lives, whether or not we
agree with them about whether it is possible to have such relationships with
technologies.
9.7 The Technological Future of Relationships
9.86 In science fiction, there is no shortage of portrayals of what the future might
be like with respect to personal relationships of different kinds. In the classic
Star Wars movies and in Star Trek, for example, robots are portrayed as being
members of the team, colleagues, and good friends of the human characters
involved. For example, C3PO and R2D2 are just as much members of the team
and close friends with Luke Skywalker as Princess Leia and Han Solo are—and
the humanoid robot Data is also seen and treated as a friend and good colleague.
9.87 In the 2013 movie Her, the male lead character Theodore falls in love with
an operating system, whereas in the 2014 movie Ex Machina the character Caleb
falls for but is ultimately abandoned by the artificially intelligent robot Ada. In
2004's Eternal Sunshine and the Spotless Mind, the female lead character played
by Kate Winslet uses an advanced memory‐erasing technology to wipe out all
memories of her romantic relationship with the male lead character, played by
Jim Carrey, so that she can move on to other relationships. Carrey's character
then tries to outsmart the memory‐erasing technology by finding a loophole back
into the memory of his former partner, so that she can remember the romantic
relationship they had together and perhaps rekindle the flame.
9.88 Those are some examples from science fiction of how the technological
future of relationships is imagined in some well‐known films and TV series.
Above, we have been looking at real‐life counterparts of these fictional
depictions of what the future of relationships might look like. Are there any
other technological developments that might also be predicted to potentially
significantly affect the future of relationships that are worth mentioning before
we wrap up this chapter? Let us finish this chapter by very briefly considering
two more examples, namely, what might be called:
“Quantified relationship technologies”: technologies that can be used to track,
log, or quantify various aspects of personal relationships. Basically, these are
akin to “self‐tracking technologies”—or what we called “behavior‐change
technologies” in Chapter 5—but applied to personal relationships. Such
technologies can be envisioned in different kinds of relationships, but have so far
primarily been developed for romantic and sexual relationships. Among other
things, these quantified relationship technologies might generate quantifiable
data about our romantic relationships or sexual experiences or even gamify
these. In so doing, these technologies—usually in the form of apps on smart
phones, but sometimes also in the form of separate devices—direct the users'
attention towards aspects of our relationships and intimate lives that are
amenable to quantification and gamification. For example, how often do we give
our partners flowers? How often do we have sex? How do we rate different
romantic partners or sexual encounters?
“Love drugs” or biomedical relationship technologies: drugs or other
technologies that would act on the biological/neurochemical dimensions of love
and other close relationships in order to influence lust, attraction, attachment, or
other aspects of our romantic lives or personal relationships. Such drugs might
be used to sustain, manipulate, or improve relationships that are worth
maintaining or developing (“pro‐love drugs”) or to facilitate the ending of a bad
relationship or one's recovery from it (“anti‐love drugs”). As things currently
stand, it is only the side‐effects of existing medications used for other purposes
that seem likely to promote such outcomes. Yet newer research into things such
as drug‐assisted couples therapy—e.g., using MDMA (also known as “ecstasy”)
or psychedelic drugs such as “magic mushrooms”—provides examples of how
chemical biotechnologies might in the future be used to affect and control our
relationships.
The former of these two examples is already a fairly widely used reality. There
are many people who use different kinds of quantification, tracking, or
gamifying technologies in the contexts of their relationships. The latter example
—“love drugs”—is something more experimental at this point in time. It is
certainly not something that is already widely used. However, the philosopher
and psychology researcher Brian Earp and the philosopher and medical doctor
Julian Savulescu have written a fascinating book called Love Drugs in which
they argue that drugs should play a significant role in the future of human
relationships.
9.89 Earp and Savulescu argue that as we are beginning to understand the
underlying biology and neurochemistry of love and personal relationships better,
we can come to have greater control over these aspects of our lives, with the help
of drugs that act on the relevant aspects of our brains. So, what they call love
drugs might also be an example of the technological future of human
relationships.15
9.90 When we take a step back and reflect ethically on these further kinds of
developments in technologies used in the context of personal relationships, how
can we best approach this reflection? One thing we can do is to make use of the
post‐phenomenological approach or “mediation theory” of technology that was
explained back in Chapter 1. With the help of that type of theorizing, we can
analyze and better understand how technologies can and do shape how we
experience personal relationships and what is possible for us within personal
relationships in terms of what our options for action are. We might then again
notice that technologies are not best seen as completely value‐neutral tools that
we use to achieve our ends, but that technologies rather shape and influence the
ways in which we value and understand things around us.
9.91 If we, for example, constantly use quantified relationship technologies to
track, log, and measure our relationships—and we increasingly start gamifying
our relationships—then this can change our focus away from more purely
qualitative and hard‐to‐quantify aspects of our relationships and toward more
quantitative and easy‐to‐measure aspects of our relationships. And if we turn
relationships into games, what we experience as a successful relationship might
also be changed—perhaps for the worse.
9.92 In other words, one key worry about quantification and gamification in the
context of relationships is that this might detract from important qualitative
aspects of personal relationships while motivating an excessive focus on aspects
of our relationships that can be measured or gamified. Moreover, by giving away
so much data about our personal lives to the technology companies that create
the apps that we might use for these purposes, we might in effect give away
much of our privacy—perhaps without realizing it—and make ourselves
vulnerable to manipulation and other forms of outside influence that we might
not welcome upon reflection. Hence these technologies might not only change
and shape how we perceive and value different aspects of our personal
relationships with other people. They might also involve a loss of privacy, and
open us up to being increasingly shaped, nudged, and influenced by technology
companies rather than by our own personal values and choices.
9.93 Questions and worries about authenticity tend to also come up when
philosophers and others reflect ethically on the just‐mentioned kinds of
relationship technologies. If we, for example, use love drugs to jump‐start, boost,
sustain, or influence our romantic relationships and if our partners do the same,
is this “true love” in the same way that it would be if our relationships did not
involve the use of drugs or other enhancement technologies? Imagine that your
romantic partner wants to use biomedical technologies to control their
relationship with you because they think that your personality and your whole
way of being are not sufficient for them to be able to maintain a relationship with
you. Would you then feel that their “love” for you is less real or less desirable
than a form of love that could be maintained without the need for love drugs?
9.94 Earp and Savulescu have many interesting things to say about such
questions in their work. And they also distinguish among many different uses
that love drugs and other relationship drugs could be put to. So, we should not be
too quick to dismiss the idea of love drugs as something that would create
inauthentic or less desirable forms of love or relationships than relationships that
do not involve the use of any such drugs. Yet, for many people, it might be hard
to not be at least initially skeptical about the idea of treating relationship
problems with the help of drugs.
9.95 Our attitudes about all of these things might change over time. Many people
today might be skeptical about things such as relationships with robots, chatbots,
or other technologies. Many people might also be skeptical about technologies
that quantify or gamify relationships, or about biomedical technologies used as
“relationship enhancements.” But in the future, perhaps all of these things will
be considered as completely normal.
9.96 In other words, in the future, there might be changes in our values in the
domain of personal relationships. There might also be changes in our values and
in our ethical convictions more generally in the future as a result of the creation
of new technologies that affect how we perceive the world and that change what
is possible for us to do as human beings. In the next and final chapter of this
book, one of the things we will do is to discuss how technologies might change
our values and our ethical convictions. We will also consider ways in which we
might come to blur the distinction between us human beings and the
technologies we use. For example, we will consider the ethics of virtual worlds
and brain implants. And we will consider whether we should adopt the sorts of
outlooks that those who call themselves transhumanists and posthumanists
adopt. But before we get there, the first thing we will do is to travel back in time
to the 1970s.
Annotated Bibliography
Betzler, M. and Löschke, J. (2021). Collegial Relationships. Ethical Theory and
Moral Practice 24 (1): 213–229.
Discusses what it is to be somebody's colleague as well as the question of
what values are characteristic of good collegial relationships.
Danaher, J. (2019). Philosophical Case for Robot Friendship. Journal of
Posthuman Studies 3 (1): 5–24.
Argues that humans and robots can be friends.
Danaher, J., Nyholm, S., and Earp, B. (2018). The Quantified Relationship.
American Journal of Bioethics 18 (2): 3–19.
This article examines eight ethical worries about using gamification, tracking,
logging, and other data‐collecting technologies in the context of romantic
relationships.
Earp, B. and Savulescu, J. (2020). Love Drugs: The Chemical Future of
Relationships. Stanford: Stanford University Press.
Investigates the possibility and desirability of creating biomedical
enhancements for intimate and romantic relationships.
Elder, A. (2017). Friendship, Robots, and Social Media. London: Routledge.
A deep‐dive into the question of whether robots can be our friends as well as
into whether social media friendships can be as deep and meaningful as off‐
line friendships.
Loh, J. (2019). Roboterethik: Eine Einführung. Frankfurt: Suhrkamp.
An overview of robot ethics, which articulates an inclusive approach to
human‐robot relationships.
Nyholm, S. and Frank, L.E. (2017). From Sex Robots to Love Robots: Is Mutual
Love With Robots Possible? In: Robot Sex: Social And Ethical Implications
(ed. J. Danaher and N. McArthur), 219–244. Cambridge, MA: MIT Press.
A critical investigation of whether mutual love between humans and robots is
possible. The book this paper is in also contains a lot of other essays related
to the ethics of sex robots, including the paper by Neil McArthur mentioned
in Chapter 8.
Misselhorn, C. (2021). Künstliche Intelligenz und Empathie. Ditzingen: Reclam.
Examines artificial intelligence and emotions, including whether AI systems
can have emotions in any sense.
Munn, N. and Weijers, D. (2022). Corporate Responsibility for the Termination
of Digital Friends. AI & Society
https://link.springer.com/article/10.1007/s00146‐021‐01276‐z.
Argues that corporations have a responsibility not to “kill” digital friends,
e.g., by terminating chatbots.
Nyholm, S. (2020). Humans and Robots: Ethics, Agency, and
Anthropomorphism. London: Rowman & Littlefield International.
Discusses, among other things, whether humans and robots can be friends, as
well as many other topics related to the ethics of human‐robot interaction.
Ryland, H. (2021). It's Friendship, Jim, but Not as We Know It A Degrees‐of‐
Friendship View of Human‐Robot Friendships. Minds and Machines 31 (3):
377–393.
This article argues that whether humans and robots can be friends is not an
either‐or matter, but is rather a matter of degree.
Notes
1 See the video “The Story of Replika, The AI App that Becomes You”:
https://www.youtube.com/watch?v=yQGqMVuAk04
2 Check out the trailer of Hi, AI! here, which features Chuck and Harmony: “Hi,
AI (Official Trailer, EN)”: https://www.youtube.com/watch?
v=0xkgk4ZkGUU
3 Carpenter talks about her book and her fascinating research in this episode of
the “Volume Podcast”: “#22 Militarized Robots with Julie Carpenter, PhD”:
https://podtail.com/podcast/volume‐podcast/‐22‐militarized‐robots‐with‐julie‐
carpenter‐ph‐d
4 See Megan Garber (2013) “Funerals for Fallen Robots” in The Atlantic, here:
https://www.theatlantic.com/technology/archive/2013/09/funerals‐for‐fallen‐
robots/279861
5 Readers who understand German might enjoy this interview with Loh about
this topic on the “Selbstbewusste KI” podcast: “Dass Roboter uns Emotionen
vorgaukeln, kann sehr wichtig sein. Im Gespräch mit Janina Loh”:
https://doi.org/10.5445/IR/1000125862
6 Here is Alexis Elder in discussion with John Danaher about friendships,
robots, and social media on the “Philosophical Disquisitions” podcast:
“Episode # 43—Elder on Friendship, Robots and Social Media”:
https://philosophicaldisquisitions.blogspot.com/2018/08/episode‐43‐elder‐on‐
friendship‐robots.html
7 For more on Aristotle's ideas about friendship, listen to this episode of the
“History of Philosophy without any Gaps” podcast: “45—The Second Self:
Aristotle on Pleasure and Friendship”:
https://historyofphilosophy.net/aristotle‐friendship
8 See the video: “Roxxxy TrueCompanion: World's First Sex Robot?”:
https://www.youtube.com/watch?v=2MeQcI77dTQ&t=1s
9 See the video: “Holly and Phillip Meet Samantha the Sex Robot| This
Morning”: https://www.youtube.com/watch?v=AqokkXoa7uE&t=2s
10 See the video: “Harmony, The First AI Sex Robot,” San Diego Union‐
Tribune, https://www.youtube.com/watch?v=0CNLEfmx6Rk
11 You can listen to the author of this book discuss “robot love” with David
Edmonds on the “Philosophy 247” podcast here: “Robot Love”:
https://philosophy247.org/podcasts/ robot‐love
12 Here's Helen Fisher's TED Talk: “The Brain in Love”:
https://www.ted.com/talks/helen_fisher_the_brain_in_love
13 You can hear Turkle discuss what she calls “the robotic moment” here:
“Sherry Turkle: Who Do We Become When We Talk to Machines?”:
https://www.youtube.com/watch?v=yYlfGc0YR3Y
14 You can hear Ryland discuss this topic with John Danaher here: “94—Robot
Friendship and Hatred”:
https://philosophicaldisquisitions.blogspot.com/2021/11/94‐ robot‐friendship‐
and‐hatred.html
15 You can listen to Earp talk about love drugs on this episode of the “Modern
Wisdom” podcast: “The Ethics of Using Drugs to Fall In & Out of Love—
Brian D. Earp|Modern Wisdom Podcast 268”:
https://www.youtube.com/watch?v=oD2‐‐X1RJsc
10
MERGING WITH THE MACHINE: THE FUTURE
OF HUMAN–TECHNOLOGY RELATIONS
10.1 The Experience Machine
10.1 In 1974, in his book Anarchy, State, and Utopia, the American philosopher
Robert Nozick imagined something that he called “the experience machine.”
This machine was part of a thought experiment, i.e., an imaginary example that
we are asked to think about in order to try to prove a point or test a hypothesis.
In this particular thought experiment, you are offered the option of being
connected to a machine that can simulate any experience you might be interested
in having. Being “hooked up” to the machine will give you more pleasant
experiences than you would be having in real life. And once connected to the
machine, you would not realize that you were so connected. Nozick described
this idea in several of his writings. Here is how he described the idea in his 1989
book The Examined Life:
Imagine a machine that could give you any experience (or sequence of
experiences) you might desire. When connected to this experience machine,
you can have the experience of writing a great poem or bringing about world
peace or loving someone and being loved in return. You can experience the
felt pleasures of these things, how they “feel from the inside”. You can
program your experiences for … the rest of your life. If your imagination is
impoverished, you can use the library of suggestions extracted from
biographies and enhanced by novelists and psychologists. You can live your
fondest dreams “from the inside”. Would you choose to do this for the rest of
your life? … Upon entering you will not remember having done this; so no
pleasures will get ruined by realizing they are machine‐produced.
(Nozick 1989: p. 104)
This type of thought experiment that Nozick discussed in his philosophical
writings has also been the subject of popular fiction, and not just philosophy
books. In David Foster Wallace's 1996 novel Infinite Jest, for example, there is a
movie that one can watch that is so entertaining and so engrossing that once one
starts watching it one gets so drawn in and so addicted to this movie that one
wants to just sit and watch it over and over again, and one forgets about
everything else.
10.2 But perhaps an even more well‐known dramatization of this type of thought
experiment is featured in the 1999 movie The Matrix. In that movie, a character
played by Keanu Reeves and all other humans are connected to machines that
create a computer‐simulated reality for them without their realizing this.
Intelligent machines have put human beings in this “experience machine” to
pacify them, so that they can use the world's resources and the humans will not
bother them. At one point, “Neo,” played by Reeves, gets a hunch that
something is going on, and is eventually offered a choice between two pills: if he
takes the red pill, he can find out about what is going on with the Matrix,
whereas if he takes the blue pill, he will return to the computer simulation and
never realize that he is living inside of a computer simulation. Which pill would
you take if you were in Neo's situation? Would you want to know the truth, even
if it might be bleak? Or would you prefer a potentially more pleasant life within
the computer simulation?
10.3 When Nozick presented his experience machine thought experiment in his
work, his aim was to argue that pleasure is not all that matters in life. In his
writings about this, he is arguing against what philosophers call “hedonism,”
which is the theory that the only thing that is ultimately worth striving for in life
is pleasure and the only ultimate evil in life is pain or suffering. Nozick thinks
that hedonism is a very limited conception of what is worth striving for in life.
10.4 As Nozick sees things, at least three things are missing for somebody who
lives inside his experience machine, and he expects that most of us would agree
with him about these three things. First, we do not only value the experience of
doing things: we want to actually do things. Second, we want to be a certain kind
of person, with certain virtues, good qualities, or excellences—we do not just
want to imagine that we are good people. Third, we want to know and be in
contact with reality, and not be misled by a faked or simulated version of reality
created by a computer simulation. We might add here, as well, that we tend to
want real relationships with others, and that we want to live meaningful lives. It
can seem doubtful that we can have these two things in a computer simulation.
10.5 As noted above, Nozick was philosophizing about this thought experiment
back in the 1970s and 1980s, and it was the subject of fiction in the 1990s.1
Now, however, with on‐going developments in virtual reality technologies, this
idea of living one's life—or part of it—inside of a computer simulation is
starting to become more than a philosophical thought experiment or pure science
fiction.
10.6 As things stand now with the current state of this kind of technology, it
might almost always be possible to tell that one is using a virtual reality headset,
for example. But as technological developments go even further than they have
gone today, perhaps it will soon be possible to have a real‐life Nozickian
experience machine. Perhaps it will soon become possible to have technologies
such that once you are using the virtual reality technology, it can (1) become
impossible to tell that you are using this technology and (2) your experiences
might be more pleasant, more exciting, and subjectively speaking more fulfilling
than the experiences you would be having if you did not use this technology. If
that were the case—or, rather, when this becomes the case—would you then
prefer to spend the rest of your life in this real virtual‐reality experience
machine?
10.7 As noted above, when Nozick wrote about this, his expectation was that
most of his readers would agree with his assessment of the desirability—or
rather undesirability—of being connected to the experience machine. Even if
real life is less pleasant than life in the experience machine, we would still prefer
real life, Nozick thought. And most people who considered his thought
experiment would tend to agree with him. But there are those who think that we
should be changing our attitudes about these matters. They think that we should
be changing our understanding of the idea of life in a virtual reality.
10.8 For example, the very influential Australian philosopher David Chalmers
argues in his 2022 book Reality + that virtual reality can be just as real as normal
reality. Furthermore, if other people are also hooked up to the same virtual
reality, we can have real relationships with them in the simulated virtual reality.
And life in this virtual reality situation can be just as meaningful as life in our
normal reality can be.2
10.9 If people were to increasingly follow Chalmers' lead regarding these
questions, and start abandoning Nozick's perspective, that might be seen as a
significant change in attitudes. Some people would see it as a bad development,
but others would see it as a good development. Those others might say that the
days of making sharp distinctions between humans and technology and between
reality and virtual reality should be considered as being over. We should merge
with the technologies we are creating. And we should merge reality with the
virtual realities that we are creating with our technologies. Or so some are
beginning to argue.
10.10 This final chapter will consider some different ideas about how we might
use technology to change ourselves, to change what it means to be a human
being, and what this might mean for the future of humanity. Since this is an
introduction to technology ethics, our main focus will be on what ethical
arguments can be used to evaluate proposals about what the future of human–
technology interaction should look like. This is a much bigger topic than what
can be covered in the brief discussion that follows in this chapter. But this book
is an introduction to technology ethics, and not a complete account of
technology ethics. So, the aim of this chapter is simply to briefly highlight some
of the work that is being done about this topic and to inspire the reader to think
more about this, and to perhaps go and read and then grapple with the work of
some of the thinkers whose views we will consider in this chapter.
10.11 A warning might be in order here. This chapter will deal with the most
speculative ideas that are discussed in this book: the potentially most unrealistic‐
seeming technologies and prospects for the future. So readers who may feel that
the previous chapters have for the most part been pretty well‐grounded in what
seems like—or what hopefully seems like—realistic visions about what
technologies can do now and in the near future should hereby be warned that this
chapter will include discussion of ideas and technologies that might seem much
less realistic, at least in the short run. The idea is that it might make sense to end
a book like this with some speculative discussion about what the future might be
like and about what ethical issues the future of technology ethics might need to
deal with. Toward the end of this chapter, though, we will bring the discussion
back down to Earth, so to speak, and end our discussion back where we started
it, in a small and rustic hut in the Black Forest of Germany, around 100 years
ago.
10.2 Different Ways of Merging with—Or Merging
with the Help of—Technology
10.12 Speaking of ideas that might be a little “out there,” it is worth mentioning
here that there are actually philosophers who take a Matrix‐inspired scenario
very seriously. That is to say, there are those who think that we should seriously
consider the prospect that we might actually already be living inside of a
computer simulation. According to this suggestion, we might be living inside a
computer simulation, without knowing it, a little bit like the humans in the
movie The Matrix. One prominent philosopher who has taken this idea seriously,
for example, is Nick Bostrom, the Swedish‐born philosopher who is leading the
“future of humanity institute” at Oxford University in the United Kingdom.
10.13 Bostrom suggests, among other things, that future people interested in
history will be interested in running many different simulations of how history
might have gone—that is, they might have an interest in so‐called counterfactual
history research. And one way of doing such research might be—at least in the
future—to run many different computer simulations of how history might have
gone. With the super‐computers that will be available in the future, it will be
possible, Bostrom takes it, to run many—as in very, very many—different such
computer simulations. And those computer simulations will be so realistic that
the people inside of them will not realize that they are simulated characters
within one of these computer simulations. How can we know, Bostrom asks, that
we are not living in one of those computer simulations? Perhaps we are!3
10.14 While Bostrom and some others—e.g., Chalmers and also the Swedish AI
expert Olle Häggström—take this simulation hypothesis very seriously, others
dismiss it as highly unrealistic, if not as downright self‐contradictory or
impossible. Here, we will not debate whether this idea make sense or whether it
is a possibility—since that is more of a question for the branch of philosophy
called “metaphysics” than it is a question for ethics. We will instead be
interested in the ethics of the idea of living in a computer simulation or, as
Nozick calls it, an experience machine as one possible way in which human
beings could merge with the technologies that we are creating.
10.15 One way that humans and technologies might “merge,” then, is if we start
living our lives or most of our lives within a computer‐simulated virtual reality.
A closely related possibility is that we could try to create a mix of real/normal
reality and virtual reality. This sometimes goes under the name “augmented
reality” technologies. For example, we could start using VR headsets that are
transparent—like a kind of goggles—that would project features onto reality, so
that what we are experiencing as we are going about our day‐to‐day life is a sort
of mix of things that are really out there and things that are simulated and
experienced by us as being part of the world.
10.16 Here we have at least three options then: living in the real world without
any virtual reality aspects to it, living in a computer‐simulated virtual reality, and
living in a partly real and partly simulated augmented reality. While some might
be skeptical about the desirability of spending most of one's time in virtual
reality, many people might be less skeptical about the mixed reality situation,
and they might view that as desirable in certain ways.
10.17 Another type of human–technology merger that is sometimes discussed is
that we could either replace certain parts of our bodies or brains with
technological devices or that we could add technological devices to our existing
bodies and brains. Medical technologies of this kind that are used by many
people include pacemakers (a device that helps the heart to function) and
different forms of brain stimulation technologies (for example, deep brain
stimulation or DBS). For instance, DBS is used, among other things, to treat
Parkinson's disease and other conditions that involve tremors. Electrodes are
implanted into patients' brains and a pulse generator sends electrical currents that
stimulate targeted brain areas. One suggestion one sometimes sees is that these
kinds of technologies should not only be used for medical purposes. We should
also use them for other purposes, such as to “enhance” ourselves.
10.18 Such possible uses might sometimes be discovered by accident, since they
might be side‐effects of what we were originally trying to achieve with these
technologies. For example, a Dutch woman who had a DBS device implanted in
order to treat obsessions and compulsions found that her DBS stimulation made
her feel very happy. The doctor was skeptical about this unintended side‐effect.
But the woman said that she had never felt so good in her life, and she wanted to
continue with her brain stimulation independently of whether or not it helped
against her obsessions and compulsions.
10.19 Many people are interested in developing different forms of brain
stimulation for non‐medical uses in addition to medical uses. For example, Elon
Musk is a driving force behind Neuralink. On their website, the Neuralink
company writes:
Neuralink is a team of exceptionally talented people. We are creating the
future of brain interfaces: building devices now that will help people with
paralysis and inventing new technologies that will expand our abilities, our
community, and our world.
(Neuralink, Expanding Our World. Available at https://neuralink.com/about/. Last accessed by July
26, 2022)
The idea here, then, is to merge our human bodies, brains, or nervous systems
with technologies not only for medical purposes, but also with the aim of
expanding our abilities, our community, and our world.
10.20 This type of idea would be different than—but could perhaps be combined
with—the idea of virtual reality computer simulations. We could use the
technologies we would put into our brains or elsewhere in our bodies to create
new abilities. But we could also conceivably use such technologies as one way
of generating virtual reality experiences or augmented reality experiences. Here,
though, in order to keep things separate in our ethical discussion, let us make a
distinction between merging with technology by having virtual reality
experiences, on the one hand, and merging with technology by putting
technologies into our bodies/brains in order to create new abilities in human
beings, on the other hand.
10.21 Let us now consider yet another idea about how we might merge with
something outside of ourselves with the help of technology. John Danaher, who
we have met before, teamed up with the philosopher Steve Petersen to argue that
we should take seriously the idea of using technology to merge our minds with
the minds of other people. The idea would be to create what Danaher and
Petersen call “hive minds.”
10.22 Danaher and Petersen note that people are already, in a way, sometimes
merging their minds when they work together in teams. This is a kind of
“rational” merging of minds, they say—because people sometimes “put their
heads together,” as the expression goes, to be able to do things they are not able
to do if they try to tackle a problem on their own. When people think and act
together, we can achieve greater things than when we try to do things on our
own.
10.23 Danaher and Petersen suggest that we should try to take this idea a step
further: we should try to develop technologies with whose help we can in a
literal sense connect our minds with the minds of other people so as to create a
combined mind. With such technologies, they suggest, we would not only think
together with other people so as to be able to come up with better ideas than we
are able to when we think and act on our own. With such technologies we would
directly share experiences, thoughts, and whatever else goes on within our
minds. We would break down the barrier between our own minds and the minds
of others. We would merge with technologies—the technologies in question—
but we would do so in order to merge our minds together with other people's
minds. Either just one other person or whole groups of people and their minds.
We might even be able to create a combined super‐mind! Or at least that's the
idea.
10.24 To also consider something more realistic: we might also use medical
technologies to merge with animals. At the time of writing, it was just reported
that the first successful pig‐to‐human heart transplant has been performed. A
man in the USA with serious heart disease received a heart transplant where the
transplanted heart was a genetically modified pig heart.4 After a few months, he
eventually died. Yet, for a while, this man was able to live with his new heart.
But in the future, genetically modified—or lab‐grown—organs might be one
way, and even a common or the most common way, to solve the problem of
organ shortages when people need an organ transplant. Perhaps we can grow or
genetically modify organs that we would use to not only treat medical problems
but also to enhance our abilities and capacities as well.
10.25 On this general topic, we might also mention here that there was a lot of
media attention back in 2015 when an Italian scientist and neurosurgeon
announced that he planned to perform a head transplant and that a Russian man
was volunteering to have his head transplanted to the body of another human
being (who presumably did not have a functioning head him‐ or herself!). That
head transplant wasn't carried out. The man who was going to have his head
transplanted onto another body had a child with his partner, and decided to not
go through with the head transplant. But the scientist who had this idea—Dr.
Sergio Canavero—is still hoping to get other volunteers. This would be quite
another way of merging with somebody else than what Danaher and Petersen
imagine when they imagine mind‐merging. And it would be quite an interesting
case to compare with that in which a man had a pig heart transplant. What other
kinds of transplants might be available in the future is something it is hard to
imagine in the present.5
10.3 Transhumanism, Posthumanism, and
Whether We Should Become—Or Perhaps
Already Are—Cyborgs
10.26 We have just seen that there are philosophers and others who think that we
should live (part) of our lives in virtual reality, that we should merge with
technologies (e.g., brain‐stimulation technologies), that we should use
technologies to merge our minds with the minds of others, and/or that we should
use technologies to make it possible to create human–animal chimeras. These
suggestions involve more or less speculative ideas about what is possible for us
to achieve with the technologies we are developing. And the authors that argue
in favor of these kinds of ideas use different, and sometimes conflicting,
arguments in favor of these ideas. Yet, there are some commonalities among the
perspectives of those who are interested in the above‐described kinds of ideas.
10.27 One such commonality is that the authors in question tend to be interested
in radical changes, developments, or transformations of different kinds: not only
with respect to the technologies we use, but sometimes also with respect to how
we think of ourselves as human beings, and also with respect to our values and
our ethical perspectives more generally. In this section, we will consider some
such perspectives. In the next section below, we will reflect more on these
perspectives, among other things by considering some possible critical responses
to them.
10.28 Some of these perspectives go under the name of “transhumanism.” This
term can refer to many different things, since there are many different kinds of
ideas that are being developed and defended under this heading. But a common
thread among those who see themselves as transhumanists is that they think that
human beings are sub‐optimal as we are, that there is room for great
improvement that can be achieved with different forms of technology, and that
we should try to optimize ourselves as human beings. We should try, in other
words, to transform ourselves into something better or more optimal than what
we currently are as human beings—hence the name transhumanism.
10.29 As noted, though, this is a “broad church,” so to speak, as there are many
different sub‐varieties of transhumanism. For example, while many
transhumanists are inspired by the utilitarian tradition of philosophy, the German
philosopher Stefan Lorenz Sorgner is a transhumanist who takes inspiration from
a very different source. Sorgner thinks that the 19th century philosophy of
Friedrich Nietzsche can serve as a theoretical basis for a transhumanist
perspective. As Sorgner sees things, Nietzsche has convincingly argued that we
should constantly question our current values, try to come up with new and
better values, and that we should also try to make ourselves into improved
versions of ourselves motivated by our new and updated values.
10.30 Those who defend transhumanist ideas based on a more utilitarian—or
more broadly consequentialist perspective—in contrast think that our current
values already motivate the idea that we should use technology to transform
ourselves into something more optimal than what we currently are. Such
consequentialist transhumanists argue that we could achieve overall better
outcomes and greater degrees of well‐being if we use technology to transform
our human nature into something more optimized. To think that our current way
of being is good enough and that there is no need for us to try to optimize
ourselves is to have a “status quo bias,” some of these transhumanist thinkers
have argued. Rather than having a bias toward the status quo, we should always
be on the lookout for ways of improving and transforming ourselves with the
help of technology.6
10.31 Another “broad church” is the philosophy of posthumanism. While
transhumanism is characterized by ideas about how to use science and modern
technologies to transform us human beings into a better version of the human
being, the posthuman school of thought—or at least some members of it—think
that we should abandon the idea of thinking of ourselves as human beings. The
science and technology scholar Donna Haraway, for example, thinks that there is
an emancipatory potential in coming to think of ourselves with the help of the
metaphor of us as a form of cyborgs.
10.32 A cyborg is a (human?) being that is partly organic and partly
technological. For Haraway, this is a metaphor that is a useful way of thinking of
ourselves. For others, it is more than a metaphor. Sorgner, for example, thinks
that we should see ourselves as cyborgs in a non‐metaphorical sense. As Sorgner
sees things—and as he puts things in the title of a recent book of his—“we have
always been cyborgs.” The idea here is that technologies of different kinds have
always helped to shape what we are as human beings. This applies both to non‐
physical tools, such as the languages we use, and to material technologies, such
as the latest AI technologies we use today.
10.33 As noted above, the label posthumanism can be understood in different
ways. For example, Janina Loh, who we have encountered in various previous
chapters, describes their own stance as what they call a form of “critical
posthumanism.” This means, among other things, that Loh thinks that we should
abandon sharp distinctions between things such as mind and body, humans and
animals, humans and technology, culture and nature, male and female, and so on
to instead adopt a more fluid and non‐binary perspective on the world. It is
limiting, Loh thinks, to cling to these kinds of distinctions in our self‐
understanding. And it is liberating to abandon them. Again, this is a suggested
change or transformation in how we think about ourselves as human beings. It is
supposed to help to open us up to new perspectives on ourselves and new
perspectives on what is desirable to strive for in life. It is supposed to be a step
away from a traditional “humanist” perspective on what it is to live a good
human life, toward a “post‐humanist” perspective instead.
10.34 Speaking of changes in perspective, when Danaher and Petersen argue in
favor of using technology to merge our minds with the minds of other people,
they admit that from the point of view of some of our current values, this might
seem like a bad idea. For example, many people, they note, are committed to
individualism: valuing ourselves as the unique individuals that we are and
celebrating individuality as a goal in itself. That ideal seems to clash with their
suggestion that we should use technologies to merge our minds with the minds
of others. In response, Danaher and Petersen argue that we may have other
values and goals that would be promoted or better achieved if we tried to merge
our minds with the minds of other people.
10.35 Moreover, as they see things, we should not cling too strongly to our
current values. We should instead have an attitude of openness to the possibility
of value change—where that value change might partly be driven by
technological developments. If we look back to the past, Danaher and Petersen
argue, we will note that human values have changed in the past. And we will
quickly note that some of the values human beings had in the past seem foreign
and sometimes immoral to us now.
10.36 In the past, for example, many people found slavery acceptable. Today, it
is almost universally considered unacceptable. If values have changed in the
past, Danaher and Petersen argue, they can change again in the future. So we
should be open to the possibility that some things that might seem foreign to us
now—e.g., merging our minds with other people's minds—might come to seem
normal to us in the future. And some of the values we currently have—e.g., our
individualism—might be abandoned in the future.
10.37 Technological developments, it is sometimes also argued, can force us to
look around for new ways of living good human lives. In a discussion of the
future of work, for example, Danaher elsewhere claims that if technologies—
robots, AI, and so on—take over most kinds of work that people do today, then
perhaps work will not be a place where we can find meaning and fulfillment
within our lives, as many people currently do. Instead, we will need to look for
other sources of value and meaningfulness in life.
10.38 One such source, Danaher suggests, could be the exploration of virtual
worlds. Another such source could be games of different kinds—which, of
course, could include games taking place within virtual reality worlds. Again,
the theme here is change, a perceived need to adapt how we see ourselves and
our lives, and perhaps also a need to update or perhaps even change our values,
so as to make ourselves as well prepared as possible for life in the technological
future of humanity.
10.4 Some Critical Reflections on the Proposals
to Merge with Technologies and the Arguments
and Outlooks Used in Favor of Such Proposals
10.39 The envisioned technologies discussed in Section 10.2—and the different
ways of merging with technologies or of merging with others with the help of
technologies—are certainly fascinating. And the philosophical perspectives and
arguments just considered in Section 10.3 are also fascinating. But there is also
room for critical reflection regarding whether these technological visions of the
future make sense. And there is room for critical reflection on the philosophical
perspectives and arguments that are being put forward in defense of the above‐
described visions of the future of human–technology relations. In this section,
we will consider some possible critical perspectives on what we have looked at
above.
10.40 Some critical perspectives that the above‐described ideas are likely to be
met with will be based on skepticism about whether the visions sketched above
about our future make sense in the first place. As noted earlier, for example,
some people who have responded to Bostrom's idea that we might already be
living in a computer simulation think that that's an idea that is somehow
incoherent, self‐contradictory, or at the very least highly unlikely to be correct.
Along the same lines, it might be doubted whether an idea such as the notion that
we should try to use technologies to merge our minds with other people's minds
makes sense. Let us briefly reflect on why that is another example of an idea that
can be hard to make clear sense of upon reflection.
10.41 One potential source of justified confusion here has to do with what we
normally take to be the relation between our phenomenal experiences and our
bodies and nervous systems. Importantly, people's phenomenal experiences are
very dependent on their particular bodies and nervous systems—including what
happens to their bodies and nervous systems in the particular environments they
are currently in. So, if our minds are fused with the help of some form of
technology, then this experiential or phenomenal mind‐fusing would presumably
have inputs from different bodies and nervous systems, which might react
differently to the environments they are in. This could potentially lead to mixed
and strange conscious states, if the idea of technologically merged or fused
minds even makes sense to begin with.
10.42 In the same way, if that head transplant that the Italian scientist Sergio
Canavero was planning to perform were to take place, the experience of the
resulting person might be very strange since normally one's brain is connected to
the rest of one's nervous system. This would be a case where a brain would be
connected with part of the nervous system of another person (namely, the person
whose body is connected to the head with the brain in it). It is hard to predict
what the experience of the resulting human being would be like. But it is even
harder to predict what the resulting experience in a “hive mind” of the sort that
Danaher and Petersen imagine would be like if we somehow were able to use
future technologies to merge our minds together with the minds of other people.
10.43 If some of those people, for example, were feeling nauseous while others
were feeling hungry, or some of the people would feel hot while others would
feel cold, or some would feel tired while others would feel energized, what
would be the resulting experience of the merged mind with inputs from all of
these different people? Hard to say.
10.44 Of course, whether or not we are able to envision what the different
technological futures for human beings that philosophers and others describe
might be like can partly depend on how limited our imagination is. If some
philosopher, some technology enthusiast, or somebody else describes some
future scenario involving some advanced technology—life in a virtual reality
world, merged minds, or whatever—then just because it might be hard to
imagine what this possibility would be like does not mean that it would not be a
desirable prospect to strive for. A problem, however, is that if we cannot fully
imagine what something would be like, it can seem that it is very hard, if not
impossible, to assess whether or not it would be a good and positive thing to try
to bring about the possibility in question.7
10.45 Another thing that comes to mind here is what is sometimes called the
“Collingridge dilemma.” This is an idea that is named after a sociologist named
David Collingridge, who discussed this idea in his 1980 book The Social Control
of Technology. The idea behind this so‐called Collingridge dilemma is that
before we know what it will be like to have a new technology in society, it is
very hard to predict the good and bad consequences or more generally good and
bad aspects of this technology. Once we do have the new technology in society,
we will come to have a much better idea of its positive and negative sides. But
by then, it might be too late to be able to do anything about the negative sides of
the technology. Because by then, its use might have become too widespread and
too entrenched in our everyday lives. The more outlandish or futuristic some
envisioned technology is—like some of the ones described in Section 10.2—the
harder it might be to predict what it would be like to have them in society, and
the harder it might be to control them if we do eventually introduce them into
society.
10.46 Let us next briefly reflect on the idea that we should not be too suspicious
about future technologies that seem undesirable to us now, since our values
might change over time and because our values have also changed over time in
the past. We saw above that thinkers like Sorgner, as well as Danaher and
Petersen, claim that we should have an attitude of openness toward the
possibility that our values will change when we develop new technologies and
that our values should perhaps also change when we develop new technologies.
10.47 In response to that idea, it might be noted that whenever we are reflecting
critically on either the past or the future, we need some set of standards for doing
that critical reflection. And it might seem that we will always have to use our
current values—or more broadly, our current ethical outlook—when we reflect
critically on either the past or the future. This can be done in at least two
different ways.
10.48 One thing we can do is to zoom in on specific values—e.g., the value of
personal autonomy or the value of friendship—and ask whether something in the
past or some future development is desirable from the point of view of this
value. Another thing we can do is to focus on either some sub‐set of our values
and ethical principles or perhaps even our whole set of different values and
ethical principles. We can then reflect on whether the thing in the past or the
potential future development appears to be desirable from the point of view of
this set of values and ethical principles. In ethical reflection, in other words, it
might seem impossible to not have a certain bias toward our current values (or
our current ethical standards more broadly) when we are considering and
evaluating something from the past, something in the present, or something that
might happen in the future.
10.49 The claim that we should not be too attached to our current perspective
since our perspective might come to change in the future can seem like an open‐
minded perspective to take, to be sure. But it can also seem like a perspective
that potentially robs us of the tools we usually use in ethical reflection: namely,
the set of values and other ethical considerations that we currently think are
important and that we use when we engage in ethical reflection.
10.50 It is possible, in other words, to become so open‐minded that we no longer
have any standards by which to judge things. We need to make use of at least a
sub‐set of our current values when we are engaging in value‐based critical
reflection on the past, the present, or the future. Our values might change in the
future, yes. But if we are now reflecting on whether that would be a good
development, the only standard or set of standards we could use to assess this
would seem to be our current values or ethical outlook. This does not mean that
we cannot reflect critically on our current outlooks. But when we do so, we also
need to make use of at least some parts of our current ways of understanding and
assessing the world.
10.51 What about transhumanism and posthumanism? What should we think
about those schools of thought? Transhumanism, we might note here, has been
called “the most dangerous idea in the world.” That phrase was used in a 2004
article in Foreign Policy, in which the sociologist and political scientist Francis
Fukuyama rather harshly criticized transhumanism. The philosophically inclined
German theologian Caroline Helmus, in turn, has argued that we should
understand transhumanism on the model of a religious worldview or outlook on
life. In her work, Helmus contrasts and compares transhumanism with the
worldview that is part of different versions of Christian theological perspectives
on the world. Her aim is to assess which of these different all‐encompassing
worldviews are more or less acceptable, or better or worse captures the human
condition.
10.52 In response to these ways of approaching transhumanism, it can be noted
again that the label “transhumanism” is used to refer to many different things,
and that there are more or less extreme forms of transhumanism. Some of these
are non‐extreme enough that perhaps most people today would count as some
form of transhumanists. Consider, for example, these remarks by Sorgner about
his own transhumanist position:
… I do believe that constant self‐overcoming is central to promoting my own
quality of life. I also consider scientific research, especially in biotechnology,
extremely important and advocate for greater sponsorship of those research
fields. I consider the availability of anesthetics, vaccinations, and antibiotics
important achievements. I hope that future achievements will also address
important challenges. This stance can be parsed as a weak form of
transhumanism.
(Sorgner 2020: p. 14)
What Sorgner here calls a “weak form of transhumanism” might strike many as a
fairly “mild” form of transhumanism. It might perhaps even be seen as not being
more than a big dose of technology optimism paired with great enthusiasm about
the potential for science to improve our lives—something that even somebody as
critical of transhumanism as Fukuyama could potentially agree with.
10.53 A lot of people would happily agree with Sorgner that constant self‐
overcoming is likely to be central to improving the quality of life for many if not
most people. They may also endorse scientific research, including
biotechnology, and also be in favor of greater sponsorship of those research
fields. And what Sorgner lists as great achievements above are certainly things
that many people would follow him in celebrating as great achievements. If
agreeing with these things is enough to count as some form of transhumanist,
then perhaps more of us are transhumanists—at least of the weak or mild type—
than we realize.
10.54 Whereas some more extreme forms of transhumanism can certainly seem
comparable to religious worldviews, as Helmus suggests, the weaker or milder
forms might perhaps less fittingly be described in such a way. But again, since
transhumanism can mean many different things—anything from techno‐
optimistic ideas inspired by utilitarianism to ideas inspired by Nietzsche, who
was definitely no utilitarian—it might be hard to take any general stance on
whether one should be in favor of or against transhumanism on a more general
level. It can seem better to consider different transhumanist positions on their
own, and to then assess their plausibility on their own terms.
10.55 What about posthumanism? And what about the idea that we should
compare ourselves to—or that we have perhaps always been—cyborgs? These
are also ideas that permit of so many different interpretations that it might seem
safest to recommend against taking any general stance on these issues. As with
transhumanism, it might be better to assess different articulations of these ideas
on their own, and to then critically reflect on whether one accepts parts or
perhaps the whole of these perspectives.
10.56 At any rate, it is not the aim of this book—since it is an introduction to
technology ethics—to take any stance on these issues. The aim is rather, as noted
above, to introduce these ideas to the reader in general terms and to inspire the
reader to perhaps go out and find other books and articles on these topics that
you might want to reflect further on.
10.5 Concluding Reflections: Revisiting the Hut
in the Black Forest
10.57 In a book with a title that is similar to the title of this book, the English
renaissance drama professor and Shakespeare expert Emma Smith makes an
interesting claim about Shakespeare's plays. The book is called This is
Shakespeare. And the claim Smith makes is that Shakespeare's plays raise more
questions than they answer. Throughout this book, we have considered suggested
answers to many different questions that are discussed within technology ethics.
Unlike Shakespeare's plays (as interpreted by Smith), articles and books about
technology ethics typically tend to suggest and argue for answers to questions
that we can ask about technology ethics. But what usually happens to the reader
is that they come away from these articles and books with more questions than
they feel that they have gotten answers.
10.58 This is partly because what philosophical articles and books about ethics
more generally—as well as about technology ethics in particular—do is to offer
arguments and reasons in favor of certain answers to ethical questions we might
be asking ourselves. They invite the reader to consider these arguments and
reasons and ultimately to make up their own mind. The well‐known historian of
ideas Quentin Skinner would say that this is exactly how it should be. In his
talks, one of the things Skinner often says is that in his role as a university
teacher, his task is to try to help his students to learn how to think. But it is not
his role to teach students what to think.8
10.59 In the previous chapters, we have considered a wide range of questions
that one can ask about technology ethics. We have also considered an even wider
range of possible answers to those questions, along with arguments in favor of as
well as against those possible answers to these questions. Here is a reminder
about some of the topics we have covered.
10.60 We started with the question of what we should understand by the word
“technology” when we are thinking about technology from an ethical point of
view. Should one always view every form of technology as a value‐neutral
means to our ends? Or are technologies not always value‐neutral, since they
might shape how we think and what we are able to do? And are there ever
circumstances when it can make sense to treat technologies as being more than
mere means?
10.61 What, for example, if a technology (e.g., a chatbot) comes to be
experienced as a friend? We have seen that while some authors insist that we
should only ever view technologies as means to our ends—and that we should
only create technologies that we can easily view as mere means to our ends—
there are also those who think that technologies can become more to us than
mere means to our ends.9
10.62 Having discussed what to understand by “technology,” we then considered
different conceptions of what ethics is about. Perhaps most importantly, we
distinguished between narrow conceptions of ethics (ethics as a list of dos and
don'ts) from broader conceptions of ethics (ethics as also involving many other
kinds of questions, e.g., questions about what it is to live a good life and how we
should understand ourselves as human beings).
10.63 This book has made use of a broad conception of ethics, and accordingly
also a broad conception of what technology ethics is and should be about.
Technology ethics has not been portrayed as only being about what technologies
should be permitted and what technologies should be forbidden. It has instead
been portrayed as being about a much wider range of questions. Of course,
questions about whether certain kinds of technologies—e.g., “killer robots”—
should be banned are very important. But they are by no means the only
questions that are discussed within contemporary technology ethics.
10.64 We have considered a wide range of methods by which questions in
technology ethics can be approached. In Chapter 3, we illustrated this plurality
of possible methods with the help of the case study of the ethics of self‐driving
cars. That is a good example of a topic within technology ethics with respect to
which a very wide range of different methods have been used.
10.65 In Chapter 4, we started with the idea of value alignment—as well as
potential misalignments—of artificial intelligence with human values. We then
broadened that topic to consider four different ways in which technologies can
more generally be seen as fitting with or clashing with ethical values.
Technologies can be seen as being good or bad in instrumental ways. But
technologies—or the ways in which they are used—can also potentially be seen
as being good or bad in themselves.
10.66 People who want to be friends with robots or chatbots, for example, seem
to treat these technologies not only as means to other ends, but as valuable in
themselves. Similarly, those who claim that robots, AI systems, or other
technologies might become able to be moral agents or moral patients—or
perhaps both—also seem to be discussing technologies in a way that suggests
that technologies or the way they function can align with non‐instrumental
values.
10.67 This raises intriguing questions about whether it is always proper to want
to be in full control over the technologies we use. So long as we view
technologies as mere means to our ends, and their value is primarily of an
instrumental kind, it seems perfectly fine to want to control technologies just as
we want to be in full control over tools we use. But if there could ever be robots,
AI systems, or other technologies that would be our friends, some sort of moral
equals, members of our teams, or whatever, then it might seem that—at least in
theory—there could be cases in which wanting to have full control over these
technologies might become a more ambiguous issue from an ethical point of
view. Again, there are people who claim that they want to have technologies as
their friends or even as their romantic partners. Would it be appropriate for such
people to at the same time want to have complete control over these
technologies?
10.68 Speaking of control, it is a topic that tends to come up whenever there are
ethical discussions of new or developing technologies. For example, we might
be attracted to some technologies because they might hold out the promise of
giving us more control over ourselves, as certain so‐called behavior change
technologies might do. But we might at the same time be afraid that other people
might become able to exercise control over us if we start using certain
technologies. This ambivalence about technologies and control was the subject
of Chapter 5. It is also something that is closely related to what we discussed in
Chapter 6: namely, questions about technologies and responsibility.
10.69 Many people worry that as technologies become more and more
autonomous and equipped with more and more powerful forms of artificial
intelligence, it might become harder to say who exactly can be held responsible
if and when something goes wrong. But as was noted in that chapter as well,
new technologies might also raise questions about who—if anybody—deserves
credit when something goes well or some good outcome is achieved.
10.70 If the role of the human being in the workplace, for example, is
diminished, and machines start taking over many tasks previously performed by
highly skilled human beings, we might end up with fewer opportunities for
achievement in the workplace. Such potential achievement gaps might be a
threat to meaningful work. How to think about human responsibility—both of
the negative and positive kinds—in an increasingly technologized world is one
of the most important topics within contemporary technology ethics.
10.71 In Chapter 7, we considered whether technologies can ever be moral
agents. That is, we considered whether any technologies can make moral
judgments, perform moral actions, and perhaps even be morally responsible for
what they do. We also tackled the question of whether it is desirable to have
machines that are tasked with making morally sensitive decisions. In Chapter 8,
we flipped things around and asked whether technologies can ever be moral
patients. In other words, can it ever be the case that it is possible to act wrongly
against any of the technologies that we are creating? For example, if we create
technologies that look and to some extent behave like human beings or like
animals, would that mean that there might be some ways in which it would be
ethically problematic to treat these humanoid robots or animal‐like technologies?
10.72 These questions about whether technologies can be moral agents or moral
patients are often thought to partly depend on whether they can be seen as
having minds of some sort. That issue is also often discussed in relation to the
topic that we discussed in Chapter 9—namely, the topic of whether technologies
can ever be our friends, perhaps even our romantic partners, or members of our
teams that we can regard as valuable colleagues. In what different ways, we also
asked, can technologies shape the future of close relationships?
10.73 Lastly, in this chapter we have considered different ideas about how we
might merge with technologies in the future—for example, by immersing
ourselves into virtual realities, by implanting brain stimulation technologies into
our heads, or perhaps by using technologies to merge our own minds with the
minds of other people. When technology ethics researchers discuss such ideas,
the topics of transhumanism and posthumanism typically come up. Accordingly,
we have briefly discussed those ideas here as well. We have also considered how
one can reflect critically on whether it would be good if our values were to
change in the future.
10.74 As noted above, we have considered lots of different answers to these
questions, and arguments in favor of as well as against those answers. The author
of this book is partial to some of those answers. As noted in the preface, this is
an opinionated introduction to technology ethics, which has not always tried to
be completely neutral. But the book has also not tried to persuade the reader that
any of the particular views that have been discussed are the right views to take
on the issues that have been discussed, so that other perspectives should be set
aside and not be reflected on any further. When it comes to questions about
philosophical and ethical matters, it is ultimately up to each person to try to
make up their own mind about what stance to take.
10.75 Can there be right or wrong answers to ethical questions? So‐called moral
realists think so. In contemporary philosophical debates, there are many different
kinds of moral realist positions. They differ in how they interpret what it means
for there to exist right and wrong answers to ethical questions. But they all agree
that there could be such things as right and wrong answers to these questions.
Others disagree. For example, the famous existentialist Jean‐Paul Sartre thought
that ethical questions do not have inherently right or wrong answers.
10.76 Yet, at the same time, Sartre thought that we all nevertheless have to take a
stance about ethical issues. And once we do make up our minds, we have to take
responsibility for our views. That is, we have to be willing to defend them and
argue in favor of them if other people challenge us and our perspectives.
According to Sartre, if we refuse to make up our minds about ethical issues, we
treat ourselves as if we are mere things, and as if we have no free will. He might
also have added that we then treat ourselves as if we were simple machines.
10.77 In other words, whether or not there can be right or wrong answers to
ethical questions, Sartre thought that we still cannot avoid grappling with them.
If we do not, we treat ourselves as mere things or machines, with no free will of
our own. Kant, in turn, thought that refusing to reflect on important topics was
equivalent to being an unenlightened and immature person. To be an enlightened
person, according to Kant, is to dare to think and try to make up one's own mind
about important questions that we face in life.10 A book like this has as its aim to
help and to stimulate the reader to think about the issues we have been looking
at. The aim is not to tell the reader what to think about technology ethics, to
paraphrase Skinner, but to illustrate how one can reflect on these issues.
10.78 There are issues within contemporary technology ethics that have not been
discussed in this book. Exciting new and developing technologies that we have
not discussed but that raise ethical questions include things such as artificial
wombs, evolutionary robotics, life‐extension technologies, neuromorphic
computing, digital twinning, technologies for space exploration, and so on and
so forth. Moreover, there are topics that we have touched on, but that could have
been discussed at greater length in this book—e.g., ethical questions about
privacy, data collection, biases, digital well‐being, value‐sensitive design, risks
and safety issues, and so on and so forth. And with respect to the questions that
have been discussed—even discussed in some detail—in the chapters preceding
this one, there is so much more that can be said than what we have been able to
say even in a relatively long book like this one.
10.79 The field of technology ethics is absolutely exploding at the moment.
There is no shortage of people who are interested in it and who are doing
research about it and publishing articles and books about it. Accordingly, this
book does not claim to have covered everything there is to say about technology
ethics—far from it. Hopefully, however, the reader will feel that they have gotten
a good idea of the sorts of things that are being discussed within technology
ethics. Again, the hope is that the reader will be hungry for more. There are so
many fascinating ethical questions that arise in the context of technology. And
there are so many on‐going technological developments that challenge our
ordinary conceptions of what it means to be a human being and to live a good
life. This is very exciting for those of us who are interested in these issues.
10.80 When one thinks about technology ethics and the fast development of the
technologies of today and of the future, one might also become slightly
overwhelmed at times. One can sometimes feel that one might want to withdraw
a little—take a break, and perhaps do what Heidegger used to do. As we saw in
Chapter 1, already around roughly 100 years ago, Heidegger felt that sometimes
he had to temporarily withdraw from all the modern technologies that were
surrounding him, to go and spend time in his simple hut deep in the Black Forest
of Southern Germany.
10.81 If you were to retreat to a hut in the Black Forest or elsewhere, like
Heidegger did, to take a break from the modern world and reflect on
philosophical questions about life, what modern technologies would you wish to
temporarily get away from and take a break from? Your phone? Social media?
Constantly being online? Your virtual reality headset, or perhaps your robotic
vacuum cleaner, if you have one?
10.82 More generally, is having more technology in our lives—and more
advanced technology—always better? Are there some parts of life where it
would be nice and perhaps better to have fewer technologies? We learned
Heidegger's answer to that question in Chapter 1. But what do you think?
10.83 As noted above, when it comes to ethical issues, each person ultimately
has to make up their own mind. That is, each person has to try to make up their
own mind unless they are content to see themselves as a mere thing or as a
simple machine. At least, that was the view that Sartre took.
10.84 Some people would here respond that we should view ourselves as being a
form of machines. Our brains are a form of computer. And our bodies are
machines that carry these computers around. Others think that there is something
fundamentally wrong with that way of viewing ourselves. Such questions about
how we should understand ourselves, and whether we should compare ourselves
to any of the technologies we use, are also part of technology ethics. They are
fascinating questions. We will not explore those questions further here, however.
It is instead time to bring this book to a close.
Annotated Bibliography
Nick Bostrom (2003). “Are You Living in a Computer Simulation?”
Philosophical Quarterly, vol. 53, no. 211, pp. 243–255.
Argues that we cannot be sure that we are not living in a computer simulation
and that it is likely that we are living in a computer simulation.
David Chalmers (2022). Reality +: Virtual Worlds and the Problems of
Philosophy, New York: Norton.
This fascinating book argues that virtual reality is genuine reality, that we can
live a meaningful life in virtual reality, and that we may even be in a virtual
world already.
John Danaher and Steve Peterson (2020). “In Defence of the Hivemind Society,”
Neuroethics, vol. 14, no. 2, pp. 253–267.
This article argues that we human beings should try to develop technologies
with which we can merge our minds with the minds of other human beings.
Donna Haraway (2016). A Cyborg Manifesto, Minneapolis: Minnesota
University Press.
Argues that we should resist sharp distinctions between humans and animals
and between humans and technologies, and that we should use the idea of a
“cyborg” as a metaphor for thinking about what it is to be a human being.
Caroline Helmus (2020). Transhumanismus—der neue (Unter‐)Gang des
Menschen? Das Menschenbild des Transhumanismus und seine
Herausforderung für die Theologische Anthropologie, Regensburg: Verlag
Friedrich Pustet.
Compares the way(s) transhumanists think about the human condition with
how Christian theologians think about the human condition.
Olya Kudina and Peter‐Paul Verbeek (2019). “Ethics from Within: Google Glass,
the Collingridge Dilemma, and the Mediated Value of Privacy,” Science,
Technology, & Human Values, vol. 44, no. 2, pp. 291–314.
An updated take on David Collingridge's famous dilemma introduced in his
well‐known 1980 book The Social Control of Technology, discussed here in
the context of smart glasses and the question of whether they pose an
unacceptable threat to privacy.
Janina Loh (2022). “Posthumanism and Ethics,” in Palgrave Handbook of
Critical Posthumanism, edited by S. Herbrechter, I. Callus, M. Rossini, M.
Grech, M. de Bruin‐Molé, C.J. Müller, London: Palgrave Macmillan.
https://doi.org/10.1007/978‐3‐030‐42681‐1_34‐1.
Articulates the form of inclusive critical humanist perspective briefly
described in the chapter above.
Robert Nozick (1989). The Examined Life: Philosophical Meditations, New
York: Simon & Schuster.
A set of philosophical essays on a variety of topics, including meaning in life
and happiness and the experience machine.
Jean‐Paul Sartre (2007). Existentialism is a Humanism New Haven: Yale
University Press.
A classic statement of an existentialist view on life.
Emma Smith (2020). This is Shakespeare, London: Penguin.
Reflects on how reading Shakespeare's plays can help us to reflect on both
big and small questions about life. Not a book about technology ethics, but a
book that shows how one can engage with ideas in creative and innovative
ways, in this case ideas from Shakespeare's plays.
Stefan Lorenz Sorgner (2020). On Transhumanism: The Most Dangerous Idea in
the World? University Park: Pennsylvania State University Press.
An overview of different forms of transhumanism, and a defense of a
Nietzsche‐inspired brand of transhumanism.
Stefan Lorenz Sorgner (2021). We have Always been Cyborgs, Bristol: Bristol
University Press.
Argues that we should understand ourselves as a form of cyborgs, and that
this should lead us to embrace transhumanism.
Notes
1 You can hear Nozick talk about the experience machine around the 35‐minute
mark in this radio interview from 1990: “Robert Nozick Interview 1990”:
https://www.youtube.com/watch?v=Ldngi2WtGik
2 You can hear Chalmers discuss these ideas in this episode of the “Future of
Life Institute” podcast: “David Chalmers on Reality +: Virtual Worlds and the
Problems of Philosophy”: https://futureoflife.org/2022/01/26/david‐chalmers‐
on‐reality‐ virtual‐worlds‐and‐the‐problems‐of‐philosophy
3 For more information about the ideas behind the simulation argument, check
out this website: https://www.simulation‐argument.com
4 “Man Gets Genetically‐Modified Pig Heart in World‐First Transplant,” BBC:
https://www.bbc.com/news/world‐us‐canada‐59944889
5 Here is a video of Sergio Canavero talking about his head transplant idea:
“Head Transplantation: The Future is Now|Dr. Sergio
Canavero|TEDxLimassol”: https://www.youtube.com/watch?
v=_EHCHv5u3O4
6 Here is an episode of the “Philosophy Bites” podcast in which Nick Bostrom
discusses the status quo bias: “Nick Bostrom on the Status Quo Bias”:
https://philosophybites.com/2012/05/nick‐bostrom‐on‐the‐status‐quo‐
bias.html
7 You can listen to the philosopher L.A. Paul discuss what she calls
“transformative experiences” on the “Mindscapes” podcast here: “85|L.A.
Paul on Transformative Experiences and Our Future Selves”:
https://www.preposterousuniverse.com/podcast/2020/02/24/85‐l‐a‐paul‐on‐
transformative‐experiences‐and‐our‐future‐selves
8 Skinner makes that claim, for example, in this video lecture about what
different ways of understanding what freedom is: “A Genealogy of Liberty: A
Lecture by Quentin Skinner”: https://www.youtube.com/watch?v=PjQ‐W2‐
fKUs
9 In June 2022, the Washington Post broke a story about a Google engineer,
Blake Lemoine, who thought that Google's language model “LaMDA” had
become a sentient or conscious person. Notably, Lemoine thought that
because this chatbot had become conscious, it should be given rights and be
treated with moral consideration. In the interview that he did with the
Washington Post, one of the things Lemoine said was the following: “I know
a person when I talk to it,” […] “It doesn't matter whether they have a brain
made of meat in their head. Or if they have a billion lines of code. I talk to
them. And I hear what they have to say, and that is how I decide what is and
isn't a person.” Other representatives of Google denied that LaMDA was
conscious, and Lemoine was put on administrative leave. Full story here:
Nitasha Tiku (2022) “The Google Engineer Who Thinks the Company's AI
has Come to Life,” Washington Post:
https://www.washingtonpost.com/technology/2022/06/11/google‐ai‐lamda‐
blake‐lemoine
10 For more information, see the Wikipedia entry “Answering the Question:
What is Enlightenment?”:
https://en.wikipedia.org/wiki/Answering_the_Question:_
What_Is_Enlightenment%3F
Index
Note: Page numbers followed by “n” refers to footnotes
academic technology ethics, 71
achievement gaps, 138, 139
advising systems, 161
aligned control, 97
AlphaGo, 127–129, 139
National Rifle Association, 4
The Analects, 33, 34
Anarchy, State, and Utopia (Nozick), 240
Ancient Chinese Confucianism, 32–36
Ancient Greece, ethics of virtue and human flourishing in, 28–32
anthropological theory of technology, 4
artificial general intelligence (AGI), 81
artificial intelligence (AI), 20, 21, 56, 77, 79–83, 149, 150, 155, 174, 258
artificial intelligence (AI)‐related technologies, 183
artificial moral agents (AMAs), 159, 163, 176
artificial wombs, 262
augmented reality technologies, 245
automated systems, 114
autonomous machines, 14
autonomous military robot, 143
autonomous mode, 53
autonomous technologies, 153
autonomous weapons systems, 25, 140
backward‐looking negative responsibility, 137, 139, 140
backward‐looking positive responsibility, 138, 144
backward‐looking responsibility gap, 142
behavioral patterns, 117
behavior‐change technologies, 21, 102, 105–110, 112, 113, 116–120, 124, 235,
259
biomedical relationship technologies, 235
Black Forest, 1–3
Black Notebooks, 3
brain implants, 23
brainstimulation technologies, 23
campaign against killer robots, 141
campaign against porn robots, 26
campaign against sex robots, 26
campaign to stop killer robots, 25
capability control, 96
collegial relationships, 228
Collingridge dilemma, 253
computer ethics, 162
Confucian ethical thinking, 33
Confucian ethics, 34, 65, 68
Confucian philosophy, 36
Confucian virtue ethics tradition, 174
consequentialism, 42
consequentialist ethical theories, 39–43
consequentialist theories, 42, 43
contractualism, 65
contractualist ethical theories, 65
contractualist justification, 69
contractualist tradition, 69
control problem, 96–99
controversial personality mode, 184
critical posthumanism, 250
data‐collecting technologies, 22
decision‐making tools, 138
deep brain stimulation (DBS), 245
deep‐sea exploration, 179
delegated control, 98, 99
destabilizing robotic arms race, 25
die Hütte, 1
drug‐assisted couples therapy, 235–236
dual use of technologies, 22
duties of care, 46, 47
effective altruism, 43n14
electronic whip, 109
emotional intelligence, 80
emotional tendencies, 162
emotions, 175
empirical ethics, 61–64
ethical accident algorithms, 70
ethical arguments, attitudes in, 63
ethical behaviorism, 18, 197, 221
ethical communities, 180
ethical decision‐making, 176
ethical impact agent, 162
ethical particularists, 165
ethical ramifications, 161
ethical technologies, 161
ethical theory, 38
ethical Turing test, 197
ethics
by analogy, 58–61
by committee, 56–58, 70
of self‐driving cars (see self‐driving cars, ethics of)
of virtue and human flourishing in Ancient Greece, 28–32
ethics by committee method, 70
ethics washing, 56
The Examined Life (Nozick), 240
explicit control, 97, 98
explicit ethical agents, 163
explicit moral agents, 170
formula of humanity, 36
forward‐looking positive responsibilities, 133, 137, 142, 144
full ethical agents, 163
full moral agents, 170
functional autonomy, 114
functionally autonomous mode, 53
gamification, 107–110
general ethical theory, 46–49
golden rule approach, 63
Golden Rule‐like formulation, 38
Golden Rule to Confucius, 37
Groundwork for the Metaphysics of Morals, 11
group agents, 149, 177
harmonious social order, 33, 34
Harmonious Technology: A Confucian Ethics of Technology, 34
hedonism, 240
hedonistic utilitarianism, 40
high‐level expert group, 56, 57
holographic virtual reality, 214
honorary citizenship, 184
How to Survive a Robot Invasion, 192
human agency, 175
human diversity, 218
human ethics, 173
human flourishing in Ancient Greece, 28–32
Humanizing Robots: How Making Humanoids Can Make Us More Human, 189
human–machine teams, 176–178
humanoid robots, 20, 185, 186
human responses, 175
human–technology relations, 252
Black Forest, 256–263
experience machine, 240–244
merging with help of, 244–248
proposals to merge with technologies, 251–256
transhumanism and posthumanism, 248–251
human‐technology teams, 174
human touch, 178
hypothetical contracting scenario, 70
implicit control, 97
implicit ethical agents, 162
independent artificial moral agents, 176–179
independent human agent, 180
independent moral agents, 178
Infinite Jest (Wallace), 240
information and communication technologies, 22
instrumentally negative misalignment of technologies, 87–90
instrumentally positive value‐alignment of technologies, 86–87
instrumental theory, 10
of technology, 3–7, 13, 17, 87, 183
instrumental value, 84
instrumental values and principles, 83–85
intelligent‐seeming technologies, 228
interdisciplinary research field, 14
Kantian ethics, 36–39, 65, 67, 68
Kantian principle, 100
King Midas problem, 89
legal theory, 71
love drugs, 235, 236
love enhancements, 215
machine ethics, 14, 151
machine ethics project, 158–160, 176
critiques of, 172–179
objections to, 163–172
and types of artificial moral agents, 160–163
machine learning techniques, 173
maximal control, 111
meaningful human control, 114, 141
mediation theory of technology, 7–11, 236
medical technologies, 22, 245
metaphysics, 245
methodological behaviorism, 197
methodologies of ethics, 51–53
military robots, 114, 176
military technologies, 22
moral agents, 14, 151, 156
machine as (see machine ethics project)
technologies regarded as, 13–16
moral community, 180, 207
moral emotions, 168
moral enhancement, 48
moral patients, robots as
humanoid robot, 186–190
moral status, 205–210
properties or abilities, 193–205
rightly or wrongly toward robots, 190–192
Tesla Bot and Erica the Robot, 182–185
moral responsibility, 131
moral status, 194
moral tribes, 180
moral zombies, 195
motivational control, 96
Nazi party, 3
negative non‐instrumental value misalignment of technologies, 94–95
negative responsibility, 132, 133
Nicomachean Ethics, 29, 220
non‐instrumental values and principles, 83–85
non‐Western traditional forms of ethics, 32
normative instrumental theory of technology, 101
nuclear war, averting, 76–79
object of control, 116, 117
opportunity costs, 144
Our Robots, Ourselves, 179
outsource responsibility, 152
persuasive technologies, 106, 108
phenomenology, 8
philosophical movement, 8
Platonic dialogs, 29
pleasure friendships, 220
porn robots, 26
positive non‐instrumental value alignment of technologies, 90–94
positive responsibility, 133
posthumanism, 255, 256, 260
post‐humanist perspective, 250
post‐phenomenological approach, 68, 236
post‐phenomenological movement, 10
post‐phenomenological school of thought, 8, 107
post‐phenomenology, 7–11
practical intelligence, 80, 81
praiseworthiness, 134
principle of utility, 40
psychopaths, 168, 169, 172, 178
quantified relationship technologies, 235, 236
The Question Concerning Technology, 3
Rawlsian accident‐algorithm, 69
reflective equilibrium method, 51, 52, 70
relational theory, 17
relationship enhancements, 238
Replika app, 213
Republic (Plato), 41
responsibility and technology, 149–152
events, 127–133
gaps, 134–148
human–machine teams and responsibility, 153–155
retribution gap, 141
reward hacking, 82
robot, 20
robotic colleagues, 227–232
Robot Rights (Gunkel), 192
robot theology, 91
romantic partners, 222–227
Roomba robot, 20
Rossum’s Universal Robots, 186
self‐control, 111, 113, 117, 121
self‐driving cars, ethics of, 53–56, 67, 80, 86, 114, 153, 159, 166, 226
empirical ethics, 61–64
ethics by analogy, 58–61
ethics by committee, 56–58
ethics of, 60, 71, 72
traditional ethical theories, 65–70
self‐tracking, 105
(self‐) tracking and data‐logging technologies, 21
self‐tracking technologies, 235
sense, think/plan, act paradigm, 20
sex robots, 26, 203, 214
social contract theories of ethics, 44
The Social Control of Technology (Collingridge), 253
social media, 9
companies, 131
status quo bias, 249
subject of control, 116
subjects and objects of control, 116–120
Superintelligence: Paths, Dangers, and Strategies, 77
Symposium (Plato), 224
technological friends, 219–222
technological future of relationships, 234–238
technological lovers, 222–227
technologies, 44–46, 133, 210, 262
Ancient Chinese Confucianism and traditional Southern African ubuntu
ethics, 32–36
campaigns, 25–28
empirical ethics, 61–64
ethics by analogy, 58–61
ethics by committee, 56–58
ethics of self‐driving cars, 53–56
ethics of virtue and human flourishing in Ancient Greece, 28–32
general ethical theory, 46–49
instrumentally negative misalignment of, 87–90
instrumentally positive value‐alignment of, 86–87
Kantian ethics, 36–39
key types of, 19–24
mediation theory of, 7–11
methodologies of ethics, 51–53
methods used in, 70–74
negative non‐instrumental value misalignment of, 94–95
positive non‐instrumental value alignment of, 90–94
regarded as moral agents, 13–16
regarded as moral patients, 16–19
traditional ethical theories, 65–70
utilitarianism and consequentialist ethical theories, 39–43
theoretical intelligence, 80
top‐down approach, 65
trace theory, 114
tracing condition, 114, 115
traditional ethical theories, 46, 48, 65–70
traditional humanist perspective, 250
traditional moral theories, 65, 67
traditional philosophical ideas, 123
traditional Southern African ubuntu ethics, 32–36, 66
traditional water, 2
transhumanism, 249, 255, 256, 260
traumatic sexual experiences, 187
trolley problem comparison, 58–61
ubuntu ethics, 65, 66, 68, 147–148, 208
ubuntu philosophy, 34, 36
ubuntu tradition, 37
uncanny valley hypothesis, 188
unity of the virtues thesis, 123
universal law formula, 38
universal moral grammar, 173
utilitarian cars, 67
utilitarian ethical theories, 39, 193
utilitarianism, 39–43
and consequentialist ethical theories, 39–43
utility friendships, 220
vacuum cleaning robot, 20
value alignment, 78, 81
problem, 79–83
value and ethical importance of control, 120–123
value‐neutral tools, 4
value sensitive design, 81
veil of ignorance, 69
virtual realities, 260
experiences, 246
virtue ethics, 68
virtue friendships, 30, 32, 36, 220
Western philosophy, 28
What We Owe to Each Other (Scanlon), 122
WILEY END USER LICENSE AGREEMENT
Go to www.wiley.com/go/eula to access Wiley’s ebook EULA.