Summary: "The Weaponization of Artificial Intelligence: What the Public Needs to be Aware of" (Dresp-Langley, 2023) Core Argument The paper argues that AI-powered Autonomous Weapon Systems (AWS) pose unprecedented threats to humanity, that the public is largely unaware of these dangers, and that international law does not exist to prohibit these weapons— not just ethical debate — is hence urgently needed to minimize the associated risks Historical Parallel The author opens with the story of Fritz Haber, whose ammonia synthesis process was a humanitarian breakthrough but later led to chemical warfare in WWI. This illustrates how scientific progress intended for good can be weaponized — the same trajectory the author sees with AI. Types of Autonomous Weapons Discussed Stationary sentry guns — auto-targeting weapons like South Korea's SGR-A1 These weapons are now integrated system that includes surveillance, tracking, firing, and voice recognition Killer robots ("slaughter bots") — systems that select and attack targets with no human input of the initial input of when to attack Drone swarms — potentially scalable to tens of thousands of drones, described as future weapons of mass destruction (Read in the article this para very carefully) Supervised autonomous weapon or “human on-the-loop” system here human beings can intervene and terminate engagements Semi-autonomous weapon or “human-in-the-loop” system here human beings can determine the targets but cannot intervene or terminate once it is launched Fully autonomous weapon or “human out-of-the-the-loop” once activated, can select and engage targets without The Discrimination Principle A central ethical objection is that all AWS, regardless of autonomy level, fail the discrimination principle from Just War Theory — the requirement to distinguish between combatants and civilians. This, the author argues, makes them ethically indefensible by definition. The Hacking Risk (Deep Dive given below) AWS are also vulnerable to adversarial hacking by non-state actors, potentially causing systems to malfunction or be turned against civilians — compounding the ethical problems with practical security ones. Policy Gaps (Deep Dive given below) The EU AI Act, while a step forward, has significant loopholes and does not explicitly regulate autonomous weapons. No international treaty equivalent to the Chemical Weapons Convention currently exists for AWS. Conclusion The author calls for greater public awareness, scientific accountability, and binding international legislation to prevent the catastrophic misuse of AI in warfare. . 1. Adversarial Hacking of Autonomous Weapon Systems: A Deep Dive What Is Adversarial Hacking? At its core, adversarial hacking in the context of AWS means that a hostile actor — whether a rival nation-state, terrorist group, or criminal network — doesn't need to physically defeat a weapon system on the battlefield. Instead, they can attack it through cyberspace, corrupting its decision-making, redirecting its targeting, or simply shutting it down. This is uniquely dangerous with AWS because, unlike a human soldier who can exercise judgment when something feels wrong, an AWS will simply execute whatever instructions its compromised system gives it. The Three Categories of Adversaries — Unpacked Category 1: Off-the-Shelf Attackers These are the most common and least sophisticated adversaries. They use readily available hacking tools — malware kits, phishing frameworks, known exploit libraries — that require little technical expertise to deploy. Think of them as opportunistic attackers who scan for known weaknesses and exploit them. While they pose the least sophisticated threat individually, their sheer volume makes them a constant background noise of risk. For an AWS connected to any network, even low-level intrusions can open doors for more serious attacks. Category 2: Resource-Rich Technical Adversaries These actors go beyond known vulnerabilities and actively hunt for new ones — so-called "zero-day" exploits that the system's developers don't even know exist yet. Nation-state intelligence agencies like Russia's GRU or China's PLA Unit 61398 fall into this category. They have the technical depth to reverseengineer hardware and software, probe systems over extended periods, and develop custom attack tools. For AWS, this means an adversary could silently compromise a system long before it is ever deployed, planting backdoors or logic bombs that activate only under specific battlefield conditions. Category 3: Nation-State Level Adversaries with Unlimited Resources This is where the threat becomes truly existential. These adversaries — think major geopolitical rivals with defense budgets in the hundreds of billions — can invest essentially unlimited time and money into developing entirely new classes of cyberweapons. They can build dedicated teams of thousands of engineers working for years on a single attack vector. The paper rightly notes that "one may not be able to imagine" the scale of resources here. Against such adversaries, no AWS can be considered truly secure, because given enough time and resources, any system can eventually be compromised. What Can Actually Be Attacked? The paper identifies several layers of an AWS that are vulnerable, each with devastating potential consequences: 1. State Information Infrastructure AWS rely on sensors — radar, cameras, LIDAR, GPS — to understand their environment. If an adversary corrupts this layer, the weapon system receives a false picture of reality. It might "see" civilian vehicles as military tanks, or fail to detect real threats entirely. This is related to a well-documented AI vulnerability called adversarial examples — where tiny, imperceptible changes to input data (like an image) cause an AI to completely misclassify what it's seeing. Applied to a weapon system, this could mean a drone swarm classifies a hospital as a military installation. 2. Communication Networks AWS in a coordinated swarm or platoon depend on constant communication — with each other, with command systems, and with human supervisors (in "human-on-the-loop" systems). An adversary who can interrupt or spoof these communications can isolate individual units, inject false commands, or cause the entire network to behave erratically. This is especially dangerous because disrupting communications in a supervised system effectively converts it from "human-on-the-loop" to "human-out-of-the-loop" — removing the one safety check the system relies on. 3. Automatic Control Algorithms This is the deepest and most dangerous attack surface. If an adversary can modify the core algorithms governing an AWS — its targeting logic, its rules of engagement, its movement protocols — they can fundamentally reprogram what the weapon does. Rather than simply disabling the system, they could redirect it against friendly forces or civilian populations. The paper describes this chilling scenario as the system being "corrupted to do what it is not supposed to" — in other words, killing the people it was meant to protect. The Cascading Network Effect One of the most underappreciated dangers the paper points to is how a compromise at any single node can propagate across the entire network. Modern AWS, especially drone swarms, operate as distributed networks — each unit shares information with others and decisions ripple across the group. An adversary who successfully compromises even one unit can potentially corrupt the network-wide state information, causing cascading failures or coordinated misbehavior across the entire swarm. This is the cyber equivalent of a single infected cell in a biological organism spreading disease throughout the body. Best Case vs. Worst Case The paper frames the outcomes starkly: Best case: The attack causes system failure — the AWS simply stops functioning. This is obviously problematic in a battlefield context but at least limits civilian harm. Worst case: The system is corrupted but continues operating — now acting as an agent of the adversary, potentially targeting civilians, friendly forces, or critical infrastructure, all while appearing to function normally. The worst case is particularly insidious because it may not be immediately apparent that the system has been compromised. Human supervisors might continue trusting its outputs, unaware that its decision-making has been fundamentally altered. Why This Undermines the Entire Case for AWS This section of the paper is strategically important because it adds a practical argument to the ethical one. Even if one were to set aside the discrimination principle entirely and argue that AWS could theoretically be programmed to distinguish civilians from combatants, adversarial hacking means that guarantee is permanently fragile. Any system that can be hacked can be turned into its opposite — a precision weapon becomes an instrument of mass civilian harm. The more autonomous the system, the greater the consequences of that compromise, because there is no human in the loop to notice something has gone wrong and intervene. This is why the paper ultimately argues that ethical debate alone cannot resolve the AWS problem — because even a perfectly ethical AWS design is only as safe as its cybersecurity, and perfect cybersecurity against Category 3 adversaries is, by the paper's own assessment, essentially unachievable. 2. POLICY GAP (DEEP DIVE) "The First Step Is Public Awareness" — A Deep Elaboration 1. Why Public Awareness Comes First Dresp-Langley's argument that public awareness is the first step is strategically deliberate. Throughout history, weapons bans have only materialized when public pressure became impossible for governments to ignore. Chemical weapons weren't banned in a vacuum — the horrifying images and accounts from WWI trenches created a groundswell of public outrage that made the Geneva Protocol politically necessary. The author is essentially saying: the same process needs to happen with AWS, but it hasn't started yet, because most people don't even know these weapons exist or are already being deployed. This is compounded by a transparency problem the paper identifies earlier — ongoing military research on AWS is largely classified and not available in the public domain. Unlike nuclear weapons, which generated massive public debate and the anti-nuclear movement, AWS development is happening quietly, in defense laboratories, away from democratic scrutiny. The public cannot pressure governments about something it doesn't know is happening. 2. The Scientific Community's Responsibility The call for scientists to "assume responsibility" is one of the paper's most pointed arguments, and it echoes a long tradition of scientific activism — from Einstein and Oppenheimer's postManhattan Project regret to today's AI researchers signing open letters against autonomous weapons. The argument has several dimensions: Scientists as inadvertent contributors. The paper makes clear that research in autonomous vehicles, robotics, drone navigation, computer vision, and AI — all civilian fields — can be directly translated into AWS capabilities. A researcher developing better object recognition for self-driving cars is, whether they intend it or not, potentially advancing targeting systems for killer drones. This creates a moral obligation that goes beyond the immediate application of one's work. Scientists as credible voices. Because much public discourse on AWS relies on "common sense rather than scientific evidence" (as the paper notes earlier), scientists are uniquely positioned to cut through misinformation and provide grounded, evidence-based perspectives on what these weapons can and cannot do, how dangerous they really are, and why existing legal frameworks are inadequate. Scientists as gatekeepers. Institutions, universities, and research bodies can choose what they publish, what they patent, and who they collaborate with. The paper implies that the scientific community needs to develop ethical frameworks around dual-use research — similar to the biosafety protocols that govern research on dangerous pathogens. 3. Why the Geneva Protocol Is Now Under Threat The paper draws a direct line from the Geneva Protocol — the landmark 1925 treaty banning chemical and biological weapons — to the current AWS problem. This comparison is carefully constructed: The Geneva Protocol worked because chemical weapons had identifiable, containable characteristics — they were manufactured at specific facilities, required specific precursor chemicals, and their use was detectable. International inspectors could verify compliance. The Chemical Weapons Convention built on this with rigorous verification mechanisms. AWS fundamentally break this model for several reasons: They are built from dual-use technologies — the same chips, sensors, and algorithms used in civilian products, making manufacturing impossible to uniquely identify or restrict Their "lethality" is not inherent in a chemical compound but in software, which can be copied, modified, and distributed invisibly A drone swarm can be assembled from commercially available components, as the paper's example of MIT students building the Perdix swarm demonstrates Their effects can mimic those of weapons of mass destruction — mass casualties, indiscriminate targeting — without the physical signatures that trigger existing treaty mechanisms So when the paper says AWS "threaten the continued effectiveness" of the Geneva Protocol, it means something precise: a nation-state or non-state actor could potentially achieve mass destruction outcomes while technically not violating any existing treaty, because no treaty yet covers this category of weapon. 4. Why Ethical Debate Alone Cannot Solve This This is perhaps the paper's most intellectually bold claim, and it deserves careful unpacking. The author isn't dismissing ethics — she's making a structural argument about the limits of ethical discourse as a policy tool. The problem is this: ethical frameworks like Just War Theory can tell us that AWS are wrong in principle, but they cannot: Compel states to stop developing them — a nation that believes AWS give it strategic advantage has every incentive to continue, regardless of ethical consensus Create legal liability for their misuse — without a treaty, there is no enforceable consequence for deploying AWS against civilians Prevent non-state actors from acquiring or building them — terrorist groups are not party to ethical debates Keep pace with technological change — ethical frameworks are retrospective, built on past experience, while AWS technology is evolving faster than any deliberative process can track History bears this out. There was broad ethical consensus that chemical weapons were monstrous before the Geneva Protocol — but they continued to be used until a binding legal prohibition existed. Ethics informed the law but could not replace it. The author is therefore calling for something analogous to the Chemical Weapons Convention but for AI-controlled weapons — a binding international treaty with verification mechanisms, enforceable penalties, and universal ratification. 5. The EU AI Act: Promise and Failure The paper's assessment of the EU AI Act is nuanced — acknowledging it as a genuine first in global governance while being clear-eyed about its inadequacies in the specific context of AWS. What it gets right conceptually: The European Commission's stated goal of ensuring AI is "human-centric and trustworthy" is philosophically aligned with everything the paper argues. The risk-pyramid approach — categorizing applications by danger level — is a sensible governance architecture in principle. Where it structurally fails: The paper identifies three specific failures that remain relevant today: The listing problem. The Act only regulates what is explicitly listed as high-risk. This creates a dangerous gap — any novel AI application that emerges after the list was written is effectively unregulated until lawmakers catch up, which could take years. In a technology domain moving as fast as AI, this is a fundamental design flaw. The military exemption. As we discussed in the previous elaboration, the Act simply does not apply to military AI systems. National security is treated as a blanket exemption. This means the very category of AI application the paper is most concerned about — lethal autonomous weapons — falls completely outside the Act's scope. The inflexibility problem. The law has no adaptive mechanism. It cannot dynamically reclassify an application as high-risk when new evidence of danger emerges. This is like a food safety law that can only regulate ingredients listed at the time of writing, with no mechanism to add new substances later. The "compromised version" problem: The paper notes that what was finally approved in December 2022 was a compromised version of the original proposal — meaning political negotiations between EU member states, many of which have their own defense industries and military interests, resulted in a weaker text than initially envisioned. This is an important reminder that regulatory outcomes are shaped by political economy, not just good intentions. 6. The Geneva Convention Gap — The Core Legal Vacuum The paper's final point is its most sobering. The Geneva Conventions and their associated protocols represent humanity's most developed framework for limiting the barbarity of war. They were built over more than a century of negotiation, informed by the horrors of two World Wars and countless smaller conflicts. But they were built for a world of human combatants making human decisions. They assume: A human pulled the trigger and can be held accountable A commander gave an order and bears legal responsibility A state deployed the weapon and can be sanctioned AWS break all three assumptions simultaneously. When a drone swarm autonomously selects and kills civilians, there is no clear answer to the question: who is responsible? The programmer who wrote the algorithm years earlier? The commander who activated the system? The state that deployed it? This attribution gap means existing international humanitarian law — however robust in other contexts — has no clear mechanism for accountability when AWS cause harm. The paper's conclusion is therefore a call to action: the international community needs to negotiate a new legal instrument — as ambitious and universal as the Chemical Weapons Convention — that specifically addresses AI-controlled autonomous weapons, before the technology outpaces any possibility of meaningful regulation. The longer this is delayed, the more entrenched the technology becomes in national defense strategies, and the harder it becomes to ban. This is, ultimately, the paper's central and most urgent message — and given what we now know about the Anthropic-Pentagon dispute and the ongoing paralysis at the UN's CCW discussions, it is a message that remains as urgent in 2026 as it was in 2023. ANTHORPIC vs. PENTAGON a study in light of this paper That's a fascinating question. Let me search for Anthropic's current policy on military and government use of its AI. This is a remarkably rich and timely comparison. Here's how the ideas in the Dresp-Langley article correlate with Anthropic's real-world stance: Strong Points of Alignment 1. The Discrimination Principle Dresp-Langley's central argument is that all AWS fail the discrimination principle — they cannot reliably distinguish civilians from combatants. Anthropic's position echoes this almost exactly. Anthropic stated that "without proper oversight, fully autonomous weapons cannot be relied upon to exercise the critical judgment that our highly trained, professional troops exhibit every day." This is essentially the same concern the paper raises from a Just War Theory framework. 2. Technology Reliability as a Core Objection The paper warns that current AI is not mature enough for lethal autonomous deployment. Anthropic's CEO Dario Amodei mirrored this directly: he argued that "frontier AI systems are simply not reliable enough to power fully autonomous weapons" and that the company would "not knowingly provide a product that puts America's warfighters and civilians at risk." 3. The Need for Human in the Loop Dresp-Langley categorizes AWS by autonomy level and warns specifically against "human out-of-the-loop" systems. Anthropic drew the same line, disallowing "fully autonomous weapon systems which, once activated, can select and engage targets with no human involvement." 4. Private Sector Responsibility The paper calls on the scientific community to assume responsibility in raising public awareness. Anthropic's refusal to yield to Pentagon pressure represents exactly that — a private AI company acting as a check on unconstrained military deployment of AI, at considerable financial and political cost to itself. Where They Diverge 1. Scope of Concern Dresp-Langley argues for a comprehensive international ban on AWS development and deployment, similar to the Chemical Weapons Convention. Anthropic's position is narrower and more pragmatic — Amodei acknowledged that "even fully autonomous weapons may prove critical for our national defense" and offered to collaborate on R&D to improve reliability, rather than opposing AWS categorically. 2. Gaps in Anthropic's Policy The article calls for universal protection of all civilians. Critics have noted that Anthropic's red lines do not rule out mass surveillance of human communities at large — only American citizens — and that the multitude of evolving uses of AI to inform or accelerate violence in ways that limit moral restraint are not mentioned in its acceptable use policy. Dresp-Langley would likely see this as insufficient. 3. Legal vs. Contractual Solutions The paper argues the problem can only be solved through international law and binding treaties, not ethical debate or private agreements. Anthropic's approach has been contractual — embedding restrictions in usage policies — which is precisely the kind of patchwork the paper warns is inadequate. The Bigger Picture What makes this comparison especially striking is that the Dresp-Langley paper (2023) essentially predicted the kind of confrontation that unfolded in 2026. The dispute culminated in an unprecedented attack on a private company by the federal government, with the Pentagon designating Anthropic a "supply chain risk to national security" — precisely because Anthropic tried to be the kind of responsible actor the paper called for. In short, Anthropic's stance is a real-world, partial embodiment of the paper's recommendations — sharing its core ethical concerns but stopping short of the sweeping international legal framework Dresp-Langley ultimately argues is necessary.
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )