KILLER ROBOTS
PHIL 2401: ETHICS OF A.I.
DR. JOSH REDSTONE
WEEK FOUR
KILLER ROBOTS
• Today we’ll examine “Killer Robots” by Robert Sparrow (2007). In this paper, Sparrow questions whether
it is ethical to employ lethal autonomous weapons systems (LAWS) in warfare.
•
Specifically, he considers “who we should hold responsible when an autonomous weapon system is involved in
an atrocity of the sort that would normally be described as a war crime” (p. 62).
• He canvases the designers of such machines, the commanders of such machines, and the machines
themselves.
•
Sparrow argues that because LAWS cannot be held justly responsible for deaths in warfare their use is
unethical.
1
ROBOTS AT WAR
• At the time of writing (2007) the United States Army’s Future Combat Systems Project was set for
deployment by 2012.
• The project continued from 2003-2009. This – and similar projects – cost about 32 billion dollars.
• The project was cancelled and we’re still a ways off from fully autonomous weapons systems.
• Nonetheless, Sparrow’s paper – and the ethical considerations therein – remain important.
• Afterall, the US military is still the largest player in terms of weapons research, and one of the largest
players in AI research.
2
ROBOTS AT WAR
•
Weapons with limited autonomy include
“cruise missiles, torpedoes, submersibles,
robots
for
urban
reconnaissance,
Uninhabited Aerial Vehicles (UAVs) and
Uninhabited Combat Aerial Vehicles (UCAVs)”
(Sparrow 2007, p. 63).
•
Pictured here is a German V-1 “flying bomb,”
a WWII-era cruise missile (photo from
Wikimedia Commons).
•
Note that these are not autonomous in the
sense discussed by Powers and Ganascia
(2020).
3
ROBOTS AT WAR
• Other examples include Boeing’s
AGM-84H/K SLAM-ER missile [still in
use], the US Air Force’s Low Cost
Autonomous Attack System (LOCAAS)
[project cancelled].
• Of course, Unmanned Aerial Vehicles
(UAVs) such as General Atomics MQ-9
Reaper (picture from Wikimedia
Commons) have seen use in many
21st century theatres of war.
4
ROBOTS AT WAR
• Atlas (above) and BigDog (below), each developed by
Boston Dynamics.
•
Images from Wikimedia Commons.
• Each was funded by the Defense Advanced Research
Projects Agency (DARPA).
• BigDog was meant to be used as a “pack mule.”
•
It was cancelled (due to noise concerns over its gasoline
engine).
• The hydraulic version of Atlas – as search and rescue
robots – was cancelled in 2024.
•
An electronic version was immediately announced to
replace it.
5
HOW AUTONOMOUS ARE AUTONOMOUS WEAPONS
SYSTEMS?
• Sparrow (2007) provides many more examples, but here is the point:
1. Autonomous and/or teleoperated weapons systems might minimize friendly casualties.
2. They are likely going to be cheaper than the “manned” versions used today.
• But how autonomous are autonomous weapons systems?
“Sometimes the claim that a weapon is ‘autonomous’ means only that it is capable of acting independently of
immediate human control; typically it means that it is a ‘fire and forget’ system capable of determining its
trajectory or pursuing its target to some limited extent. Many existing autonomous weapon systems are of this
type. Weapons of this nature do not in themselves raise any ethical questions beyond those raised by other
modern long range weapons” (Sparrow 2007, p. 65).
•
Powers and Ganascia (2020) would call these systems “automatic” rather than “autonomous.”
6
HOW AUTONOMOUS ARE AUTONOMOUS WEAPONS
SYSTEMS?
• Autonomy cont’d…
“However, much more than that is being claimed for the next generation of intelligent robots. According to a
number of writers the robots of the future will be capable of acting on their own, in some more robust sense.
Artificially intelligent weapon systems will thus be capable of making their own decisions, for instance, about their
target, or their approach to their target, and of doing so in an ‘intelligent’ fashion. While they will be programmed
to make decisions according to certain rules […] the actions of these machines will be based on reasons, but these
reasons will be responsive to the internal states — ‘desires’, ‘beliefs’ and ‘values’ — of the system itself” (Sparrow
2007, p. 65).
•
Such weapons systems would really be autonomous.
•
They’d also arguably be intelligence, and capable of learning, if so designed.
•
On danger, Sparrow argues, is that such machines will operate in unpredictable ways.
7
HOW AUTONOMOUS ARE AUTONOMOUS WEAPONS
SYSTEMS?
• Autonomy cont’d…
•
So, such systems have a claim to autonomy; moreover, with autonomy comes moral responsibility.
“To say of an agent that they are autonomous is to say that their actions originate in them and reflect their ends.
Furthermore, in a fully autonomous agent, these ends are ends that they have themselves, in some sense, chosen”
(Sparrow 2007, p. 65).
•
This is agency in the philosophical sense!
•
If some action A originates from an agent S, in the sense that A originates from S’s internals states (beliefs,
desires, etc.) and her chosen ends, then S can be held accountable for A.
•
Can we think of some examples?
•
Note that Sparrow is conservative about his speculations of robot autonomy. Robots may not achieve a fully
human degree of autonomy…but they may be much more autonomous that current weapons systems.
•
Yet, we should probably make ourselves aware of – and try to contend with – the emerging ethical issues
before this degree of autonomy is achieved.
8
ROBOT WARRIORS AND ROBOT WAR CRIMES?
• An abundance of ethical questions is raised here:
“How will the use of robot weapons affect the ways in which wars are fought, the level and nature of casualties,
and the threshold of conflict? What sort of decisions should they be allowed to make? How should they be
programmed to make them? Should we grant non-human intelligent agents control of powerful weapons at all? If a
system is intelligent enough to be trusted with substantial decisions making responsibility in battle, should it also
be granted moral standing? Should they then be granted rights under the Geneva Conventions?” (Sparrow 2007, p.
66).
•
The question we’re interested in is: “who should be held responsible if an AWS was involved in a wartime
atrocity of the sort that would normally be described as a war crime[?]” (Ibid.).
•
This question, Sparrow says, is prior to all the others.
•
If it turns out that no one can be justly held responsible, then the use of autonomous weapons systems is
unethical, and we shouldn’t use them.
9
ROBOT WARRIORS AND ROBOT WAR CRIMES?
• Example War Crime:
“Let us imagine that an airborne AWS, directed by a sophisticated artificial intelligence, deliberately bombs a
column of enemy soldiers who have clearly indicated their desire to surrender. These soldiers have laid down their
weapons and pose no immediate threat to friendly forces or non-combatants. Let us also stipulate that this
bombing was not a mistake; there was no targeting error, no confusion in the machine’s orders, etc. It was a
decision taken by the AWS with full knowledge of the situation and the likely consequences. Indeed, let us include
in the description of the case, that the AWS had reasons for what it did; perhaps it killed them because it calculated
that the military costs of watching over them and keeping them prisoner were too high, perhaps to strike fear into
the hearts of onlooking combatants, perhaps to test its weapon systems, or because the robot was seeking to
revenge the ‘deaths’ of robot comrades recently destroyed in battle. However whatever the reasons, they were
not the sort to morally justify the action. Had a human being committed the act, they would immediately be
charged with a war crime” (Sparrow 2007, p. 66).
10
ROBOT WARRIORS AND ROBOT WAR CRIMES?
• Responsibility and Jus in Bello:
•
Jus in Bello: from the Latin for law/right/justice in war; refers to the international humanitarian laws that
govern conduct during armed conflict.
•
“The question of the attribution of responsibility in this situation matters because I take it that it is a
fundamental condition of fighting a just war that someone may be held responsible for the deaths of enemies
killed in the course of it. In particular, someone must be able to be held responsible for civilian deaths. The
responsibility at issue here is moral and legal responsibility and not mere causal responsibility” (Sparrow 2007,
p. 67).
•
An important assumption is that agents in war can be held responsible. Otherwise, no Jus in Bello.
•
The idea that we can identify and hold responsible those who cause deaths in war accords with deontological
and consequentialist theories.
11
ROBOT WARRIORS AND ROBOT WAR CRIMES?
• Responsibility cont’d…
“If the nature of a weapon, or other means of war fighting, is such that it is typically impossible to identify or hold
individuals responsible for the casualties that it causes then it is contrary to this important requirement of jus in
bello. It will not be ethical to employ this means in war” (Sparrow 2007, p. 67).
•
Are LAWS more similar to, e.g., firearms and attack vehicles? Or, to weapons of mass destruction or antipersonnel mines? Do you see the difference?
• Will Insisting on “Human Oversight” Avoid the Problem?
•
What if we required human officers to approve any actions that LAWS might take that could end human lives?
•
Sparrow argues that this might work as a near-term solution; eventually, however, these decisions will be
handed over to machines. Why?
12
ROBOT WARRIORS AND ROBOT WAR CRIMES?
• Human Oversight cont’d…
•
What if human operators become redundant or even disadvantageous?
•
What about military strikes against the infrastructure that allows human operation/oversight?
•
“It seems likely that sometime in the not-too distant future, the time available to make survival critical
decisions will often be less than the time required for a human being to make them. When this occurs, then
only robots will be capable of participating effectively in these forms of combat” (Sparrow 2007, p. 68).
•
Sparrow argues that such reasons mean autonomous weapons will need to operate fully autonomously.
•
A version of the question of responsibility emerges here anyway. AI is used to analyze, synthesize, and interpret
information that human combatants rely upon.
•
If a human – a fighter pilot, say – acts upon information that is wrong, and the rules of war are broken, then
who do we hold responsible?
13
RESPONSIBILITY FOR ROBOT WAR CRIMES?
“Who should we hold responsible for a war crime in a situation that crucially involves a decision made by
an AWS, such as that described above?” (Sparrow 2007, p. 69).
• The Programmer?
•
If a machine committed a war crime, could we hold the programmer responsible if the machine’s action is due
to negligence/error on the part of the programmers?
•
Maybe not…
•
The machine’s actions could be the result of a limitation of the system. If the programmers acknowledge such
limitations but the system is employed anyway, then the programmers are not responsible.
•
The machine’s actions could be the result of its own autonomy, i.e., it behaves in an unpredictable way.
•
In the first case, we can hold the commanding officer(s) responsible. In the second case, can we hold the
machine responsible?
14
RESPONSIBILITY FOR ROBOT WAR CRIMES?
• The Commanding Officer?
•
Perhaps, then, we’d best hold those responsible for ordering the use of the machine?
“[This] accords with the precedent set by our response to cases where other weapons that may kill people other
than their intended target are used. In these cases we simply insist that those who use them should be held
responsible for the deaths they cause, even where these were not intended” (Sparrow 2007, p. 70).
•
There are parallels with, e.g., long-range artillery fire. But these are ‘dumb’ weapons. What about ‘smart’
weapons?
“[W]e normally think that smart bombs, and other AWS, are more reliable ways to attack the enemy. It is hoped
that future AWS will even be capable of discriminating reliably between civilian and military targets. This has even
led a number of critics to argue that the use of these weapons is morally superior to the use of ordinary ‘dumb’
weapons” (Ibid.).
15
RESPONSIBILITY FOR ROBOT WAR CRIMES?
• The Commanding Officer?
“At this stage of my discussion it will have to suffice to note that the autonomy of the machine implies that its
orders do not determine (although they obviously influence) its actions. The use of autonomous weapons
therefore involves a risk that military personnel will be held responsible for the actions of machines whose
decisions they did not control. The more autonomous the systems are, the larger this risk looms. At some point,
then, it will no longer be fair to hold the Commanding Officer responsible for the actions of the machine. If the
machines are really choosing their own targets then we cannot hold the Commanding Officer responsible for the
deaths that ensue” (Sparrow 2007, p. 71).
16
RESPONSIBILITY FOR ROBOT WAR CRIMES?
• The Machines?
•
It’s reasonable to regard a machine as causally responsible; but morally responsible?
•
One problem: how do we hold a machine responsible (in the moral sense)? That is, is it conceptually possible to
reward or punish a machine?
•
What do you think? Do you think a machine that is sufficiently sophisticated could be rewarded or punished?
“In order for a machine to be capable of being punished […] it must be possible for it to be said to suffer.
Furthermore, its suffering must be of the sort that we find morally compelling” (Sparrow 2007, p. 72).
•
Moreover, this suffering must be genuine; it must motivate the machine to respond to punishment in the same
way that punishing another human would.
•
Maybe the machine would even need moral emotions?
“Indeed, the suffering involved in a machine being punished must be such that if we discover that it was in fact
innocent (perhaps the bomb that killed the soldiers was launched by another AWS near by) then we feel that we
have done it a serious wrong and owe it recompense” (Ibid.).
17
RESPONSIBILITY FOR ROBOT WAR CRIMES?
• The Machines?
“What must be true in order for a machine to be held morally responsible is therefore much more demanding than
at first appeared. Not only must machines have internal states of equivalent complexity and with analogous
structure to those of human beings, but they must also have the capacity to express these in ways that will
establish the moral reality of these states. They must make the same sorts of moral and empathic demands upon
us as do other (human) people” (Sparrow 2007, p. 72).
•
This is what many optimists hope for, but others are skeptical that this can be achieved.
•
Also, achieving such machines would be paradoxical: the idea is that robot soldiers fight so that human soldiers
don’t risk their lives and/or well-being.
•
But if a machine could suffer, then we ought to be just as morally concerned about the machines as we are
human soldiers.
•
So, where do we go from here?
18
RESPONSIBILITY FOR ROBOT WAR CRIMES?
• Robot Warriors and Child Soldiers:
“We can better conceptualize this dilemma if we consider another case where attribution of responsibility in
wartime is problematic; the use of child soldiers in war” (Sparrow 2007, p. 73).
•
Like LAWS, child soldiers have some degree of autonomy, but not sufficient moral agency/autonomy to be held
accountable for their actions.
•
Further, they are sufficiently autonomous that their actions may be unpredictable (like LAWS).
“It is the prospect of intelligent actors without any moral responsibility that makes child armies especially
terrifying. When child armies take to the battlefield, as they have in Angola and Liberia in recent years, no one is in
control. If civilians are killed they are killed senselessly without anyone being responsible for their deaths. The
deaths that occur, occur, in a sense, indiscriminately — without necessarily being random” (Ibid., p. 74).
19
RESPONSIBILITY FOR ROBOT WAR CRIMES?
• Robot Warriors and Child Soldiers:
•
Thus: “There seems to be a conceptual space in which children and (perhaps) machines are sufficiently
autonomous to make the attribution of responsibility to an appropriate adult problematic, but not so
autonomous as to be responsible for their own actions” (Sparrow 2007, p. 74).
• Conclusion?
•
It is unethical to employ lethal autonomous weapons systems (LAWS) in warfare. Why?
•
The machines cannot be held responsible.
•
Commanding officers may be held responsible for actions machines take on their own.
•
Hence the parallel with child soldiery.
20
KILLER ROBOTS
• Those who are interested should watch this film: Hood. (2015).
Eye in the Sky [Film]. Entertainment One Ltd.
• It depicts the efforts of the British, American, and Kenyan
militaries using some of technology discussed today in order to
conduct a precision strike against members of Al-Shabaab.
• I won’t spoil anything…but the film provides an interesting look at
the use of smart weapons, AI in the military, the chain of
command, collateral damages, etc.
• A good film to inspire more thinking about killer robots!
21
NEXT TIME…
• Quiz #1 takes place on Tuesday, February 4th.
• I will post an announcement about the details of the quiz on Brightspace.
• Then, we’ll read John Danaher’s chapter: “Sexuality” (Course textbook, chapter 21).
22