The change on Moral Perception
in relation to increased
autonomous systems
Experimental Proposals
by Tjerk de Greef & Alex Leveringhaus
Challenge the future
1
What is moral perception?
• Denotes the discernment (the ability to judge well) of morally
relevant issues in particular situations
• Moral perception informs our moral reasoning, which is the
deliberation of what is the right thing to do
• In international law and just war theory, moral perception
plays a key role:
• Accurate perception of conflict situation & knowledge of relevant
regulation/principles. Ability to apply correct principles to
particular combat situation.
• Determine what course of action is necessary.
• Determine whether a target is a legitimate target/combatant
(in other words: operator must weight discrimination, necessity
and proportionality for sound targets)
Challenge the future
2
Increased Autonomous Systems,
like drones, are…
Perfect for the dull, dangerous, dirty, and dodgy tasks
BUT
Potentially induce risks in moral perception, decision making,
and action taking
THEREFORE
Experimental studies required
Challenge the future
3
Questions?
• Do we fully understand the psychological effects on remote
operators of conducting war at a distance?
• What are the psychological effects if we extend the concept
of ‘war at a distance’ to reduced interaction (i.e., increased
autonomy) with drones?
Challenge the future
4
Characterization of Increased
autonomous systems
1. Distance from the battlefield -> due to controlling a targeting
system from a large distance -> decreased risk for the operator
2. Distance from the battlefield -> reduction of battlefield
stressors -> leads to better moral judgments
3. Increased surveillance capabilities-> due to more sensors
4. Increased surveillance capabilities -> due to better sensors
5. Increased surveillance endurance-> due to increased flying
time (up to 36 hours)
6. (2) + (3) + (4) -> reduced collateral damage due to improved
recognition of patterns/improved situation awareness
7. (2) + (3) + (4) introduced a bonding risk with object/actors in
the area of interest
Challenge the future
5
Drones vs. Traditional Targeting
Systems
1. Distance from the battlefield -> due to controlling a targeting
system from a large distance -> decreased risk for the operator
2. Distance from the battlefield -> reduction of battlefield
stressors -> leads to better moral judgments
3. Increased surveillance capabilities-> due to more sensors
4. Increased surveillance capabilities -> due to better sensors
5. Increased surveillance endurance-> due to increased flying
time (up to 36 hours)
6. (2) + (3) + (4) -> reduced collateral damage due to improved
recognition of patterns/improved situation awareness
7. (2) + (3) + (4) introduced a bonding risk with object/actors in
the area of interest
Challenge the future
6
However, drones have downsides
• Soldiers are removed from the horrors of war and see the
enemy not as humans but as blips on a screen, there is very
real danger of losing the deterrent that such horrors provide
(cf. game mentality)
• Are operators becoming trigger happy with remote controlled
armaments, situated as they are in complete safety, distant
from the conflict zone (cf. incorrect judgment of necessity
principle)
• Not so precise as predicted leading to increased collateral
damage
Challenge the future
7
Assumptions
• Focus in the usage of targeting systems in bello and not in
the justice in the declaration of war (jus ad bellum)
Challenge the future
8
Research Question in experiment
• How is moral perception affected by tele-operating a UAV?
• Specific judgements: necessity, proportionality,
discrimination.
Challenge the future
9
Two separate experiments
1. Distance in manipulated -> studying moral perception
2. Drone Endurance & Sensor Quality/type are manipulated
-> studying quality of attack and bounding with object/actors
• Preferably using same environment
Challenge the future
10
Experiment
• While tele-operating a system, systematically manipulate
• Proximity
• In the field with direct sight
• At a large distance with no direct sight
• Stressors/unexpected situations (or some other intervening
variable)
• Many
• Little
• Type of Task
• Surveillance & Reconnaissance
• Engagement
• Using naïve (non-professional) subjects (Delft University)
Challenge the future
11
What we measure, for example
•
•
•
•
Experienced stress
Experienced risk
# of engagements
After each engagement, freeze and question about necessity,
proportionality and discrimination
• After each engagement, check whether order/action fits
RoE/LoAC
• Ask for rationale for specific actions
Challenge the future
12