Lethal autonomous weapon systems: the stages of their development, their massive topicality. How not to ban them?

By Rebecca Gordon

Here's a possible scenario to consider: A military force has purchased a million cheap, disposable drones, the size of a deck of cards, each capable of carrying three grams of explosives — enough to kill a single person or, in the case of a "shaped charge", piercing a steel wall. They have been programmed to seek out and "engage" (kill) certain human beings, based on specific "signature" characteristics like carrying a weapon, for example, or a particular skin color. They fit in a single shipping container and can be deployed remotely. Once launched, they fly and kill autonomously without any further human intervention.

Science fiction? Not really. It could happen tomorrow. The technology already exists.

In fact, LAWS-lethal autonomous weapons systems have a long history. In the spring of 1972, I spent a few days occupying the physics building at Columbia University in New York. Along with a hundred other students, I slept on the floor, ate free takeout, and listened to Allen Ginsberg [1926-1997, Beat Generation founder, poet, pacifist, and Buddhist] when he came to honor us of some of his improvised poems. I was writing leaflets then, requisitioning a Xerox machine to print them.

And why, of all the buildings on campus, did we choose the one that houses the physics department? The answer: To convince five Columbia faculty physicists to sever ties with the Pentagon's Jason Defense Advisory Group, a program offering money and labs to support basic science research that might prove useful to efforts to United States war. Our specific challenge: the involvement of Jason Defense Advisory scientists in designing parts of what was then known as the “automated battlefield” for deployment in Vietnam. This system indeed proved to be a precursor to the lethal autonomous weapons systems that are about to become a potentially significant part of the United States arsenal.

The first (semi-)autonomous weapons

Washington faced a number of strategic problems in pursuing its war in Indochina , with a marked accentuation from 1965], notably the general corruption and unpopularity of the South Vietnamese regime which he supported. His greatest military challenge, however, was probably North Vietnam's continued penetration of fighters and supplies using what was termed the Ho Chi Minh Trail. It ran from north to south along the Cambodian and Laotian borders. The Ho Chi Minh Trail was actually a network of easily repairable [after bombing] dirt roads and tracks, streams and rivers, located under a thick cover of jungle that made it nearly impossible to detect movement from the air.

The US response, developed by Jason Defense Advisory in 1966 and deployed the following year, was an attempt to ban these transports by creating an automated battlefield consisting of four elements, analogous to eyes, nerves, brain and to members of a human body. The eyes were made up of a wide variety of sensors – acoustic, seismic, even chemical (to detect human urine) – most of them air-dropped in the jungle. The nerve equivalents transmitted the signals to the "brain". However, since the sensors only had a maximum range of about 30 km, the United States military had to constantly fly planes above the foliage to pick up any signal that might be triggered by passing troops. or North Vietnamese transport. The planes then had to relay the signals to the brain [command center with high technology]. Originally intended to be remote-controlled, these aircraft performed so poorly that pilots were usually needed.

And that “brain,” a magnificent military facility built in secret in Nakhon Phanom, Thailand, housed two state-of-the-art IBM mainframes. A small army of programmers wrote and re-wrote the code to make them work, all the while trying to make sense of the stream of data transmitted by these planes. The coordinates of the targets they obtained were then transmitted to the attack planes, which were the equivalent of the limbs [according to the analogy with a human body]. The group that ran this automated battlefield was called Task Force Alpha and the whole project was codenamed Igloo White [Operation Igloo White was conducted from January 1968 to February 1973: it was designed as electronic warfare ; it was to replace Operation Rolling Thunder, i.e. the bombing campaign against North Vietnam and Laos from March 1965 to November 1968].

As it turned out, Igloo White was largely a costly failure: its price tag, about $1 billion a year for five years (or nearly $40 billion in total in today's dollars). The lag between triggering a sensor and releasing the ammunition rendered the system ineffective. Therefore, Task Force Alpha sometimes simply bombed areas where a single sensor might have triggered. The North Vietnamese quickly figured out how these sensors worked and devised methods to fool them, ranging from broadcasting truck ignition records to setting up buckets of urine.

Given the history of semi-automated weapon systems such as drones and "smart bombs" in the years that followed, you probably won't be surprised to learn that this early automated battlefield couldn't do the distinction between soldiers and civilians. In this, they have only continued a trend that has existed since at least the 18th century, whereby wars regularly kill more civilians than combatants.

None of these shortcomings have stopped Defense Department officials from viewing the automated battlefield with admiration. Andrew Cockburn described this adoring attitude in his book Kill Chain: The Rise of the High-Tech Assassins [Picador USA, 2016], quoting Leonard Sullivan, a senior Pentagon official who visited Vietnam in 1968: "Just as it's almost impossible to be agnostic in the Cathedral of Notre Dame, it's hard not to be swept away by the beauty and majesty of the temple of Task Force Alpha.” One may wonder who or what in such a temple was to be worshipped?

***

Most aspects of this Vietnam-era “automated” battlefield actually required human intervention. Humans installed the sensors, programmed the computers, flew the planes and dropped the bombs. In what sense, then, was this battlefield “automated”? As a harbinger of what was to come, the system had eliminated human intervention at one crucial point in the process: the decision to kill. On this automated battlefield, computers decided where and when to drop the bombs.

In 1969, Army Chief of Staff William Westmoreland [1914-2005] expressed his enthusiasm for eliminating the disorderly human element in the execution of warfare. Addressing a luncheon of the Association of the US Army, a pro-military and armaments lobby, he said:

“On the battlefield of the future, enemy forces will be located, tracked, and targeted almost instantaneously through the use of data links, computer-aided intelligence assessment, and automated fire control. With probabilities – approaching certainty – of killing in the first salvo, and with surveillance devices capable of constantly tracking the enemy, the need for large forces to “fix the opposition” will therefore be less important. .”

What Westmoreland meant by "fixing the opposition" was killing the enemy. Another 21st century military euphemism is “engage.” In both cases, the meaning is the same: the role of lethal autonomous weapon systems is to automatically spot and kill human beings, without human intervention.

New LAWS for a New Era: Lethal Autonomous Weapons Systems

Each autumn the British Broadcasting Corporation (BBC) sponsors a series of four lectures given by an expert in an important field of study. In 2021, the BBC invited Stuart Russell, professor of computer science and founder of the Center for Human-Compatible Artificial Intelligence at the University of California, Berkeley, to give these "Reith Lectures". Its general theme was the future of artificial intelligence (AI), and the second lecture was titled “The future role of AI in warfare”. He addressed the issue of Lethal Autonomous Weapons Systems, or LAWS, which the United Nations defines as "weapons that locate, select and strike human targets without human supervision."

Stuart Russell's main point, eloquently presented, was that while many people think that lethal autonomous weapons are a potential future nightmare, something in the realm of science fiction, “they are not. You can buy them today. They are presented on the web.

I've never seen any of the films in the Terminator series, but apparently military planners and their PR people assume that most people derive their understanding of these LAWS from this fictional dystopian world. Pentagon officials are often at pains to explain why the weapons they are developing are not, in fact, real world equivalents of SkyNet – the global communications network that in these Terminator films becomes self-aware and decides to eliminate humanity. Don't worry, as one Deputy Secretary of Defense told Stuart Russell: "We have listened carefully to these arguments and my experts have assured me that there is no risk of accidentally creating SkyNet."

However, Stuart Russell's reasoning was that a weapons system need not be self-aware to act autonomously or to pose a threat to innocent human beings. What it needs is:

***

The reality is that such systems already exist. Indeed, a Turkish state-owned arms company recently advertised its Kargu drone – a “dish-sized” quadcopter, as Stuart Russell described it, which can carry a kilogram of explosives and is capable of performing "autonomous anti-personnel strikes" with "targets selected on imagery and by facial recognition". The company's site has since been changed to emphasize its adherence to a supposed "man in the loop [decision]" principle. However, the UN reported that a fully autonomous Kargu-2 was, in fact, deployed to Libya in 2020.

You can buy your own quadcopter right now on Amazon, although you'll still need to have some DIY computer skills if you want it to operate autonomously.

In reality, lethal autonomous weapons systems look less like a Terminator movie and more like swarms of tiny killer robots. The miniaturization of computers means that the technology already exists to create effective LAWS. If your smartphone could fly, it could be an autonomous weapon. Newer phones use facial recognition software to "decide" whether to allow access. It is not difficult to create flying weapons the size of a phone, programmed to "decide" to attack specific individuals, or individuals with specific characteristics. Indeed, it is likely that such weapons already exist.

Can LAWS be banned?

So what's wrong with the LAWS, and is there any point in trying to outlaw them? Some opponents say the problem is that they eliminate human responsibility in making lethal decisions. These reviews suggest that, unlike a human being who aims and pulls the trigger on a rifle, a LAWS can choose and shoot their own targets. Therein, they say, lies the specific danger of these systems, which will inevitably make mistakes, as anyone whose iPhone has refused to recognize their face will recognize.

In my opinion, the problem is not that autonomous systems exclude human beings from lethal decisions. To the extent that weapons of this type make mistakes, human beings will always bear moral responsibility for the deployment of these imperfect lethal systems. LAWS are designed and deployed by humans, who therefore remain responsible for their effects. Like the semi-autonomous drones of the moment (often piloted from the other side of the world), lethal autonomous weapons systems do not remove human moral responsibility. They simply increase the distance between the killer and the target.

Moreover, like weapons already banned, including chemical and biological weapons, these systems have the ability to kill indiscriminately. Even if they do not remove human responsibility, once activated they will certainly escape human control, much like poison gas or "weaponized viruses" [military].

And as with chemical, biological and nuclear weapons, their use could be effectively prevented by international law and treaties. It is true that rogue actors, such as the Assad regime in Syria or the United States military in the Iraqi city of Fallujah, may occasionally violate these rules, but overall the bans on using certain types of potentially devastating weapons have been respected, in some cases for over a century.

Some United States defense experts argue that since adversaries will inevitably develop LAWS, common sense dictates that "our country" should do the same, implying that the best defense against a given weapons system is a identical system. It makes as much sense as fighting fire with fire when in most cases using water is by far the better option.

The Convention on Certain Conventional Weapons

The area of ​​international law that governs respect for human beings in times of war is, for historical reasons, called international humanitarian law (IHL). In 1995, the United States ratified an addition to IHL: the 1980 United Nations Convention on Certain Conventional Weapons. (Its full title is much longer, but its name is usually abbreviated as CCW-Convention on Certain Conventional Weapons.) It regulates the use of, for example, incendiary weapons like napalm [gelled gasoline], as well as biological and chemical agents.

CCW signatories meet periodically to discuss other weapons that may fall under its jurisdiction and prohibitions, including LAWS. The most recent conference took place in December 2021. While there are transcripts of the proceedings, only a draft outcome document – ​​produced before the conference opened – has been published. This can be explained by the fact that no consensus has even been reached on how to define these systems, let alone on whether they should be banned. The European Union, the United Nations, at least 50 signatory countries and (according to polls) the majority of the world's population believe that lethal autonomous weapons systems should be banned. The United States, Israel, the United Kingdom and Russia disagree, along with a few other countries.

Prior to these CCW meetings, a Governmental Experts Group (GEG) met, ostensibly to provide technical advice for decisions to be taken by the “high contracting parties” to the Convention. In 2021, the GEG was unable to reach consensus on banning these weapons. The United States felt that even the definition of a lethal autonomous weapon was not necessary (perhaps because if they could be defined, they could be prohibited). The United States delegation expressed itself as follows:

“The United States explained its view that a working definition should not be drafted to describe weapons that should be banned. As some colleagues have already pointed out, it would be very difficult to reach a consensus on this point, and it would be counterproductive. Because there is nothing inherent in autonomous capabilities that would make a weapon prohibited under IHL, we are not convinced that banning weapons based on degrees of autonomy, as have suggested by our French colleagues, which is a useful approach.”

The United States delegation was also keen to eliminate any wording that might require "human control" of these weapon systems:

“[In] our view, IHL does not establish a requirement of 'human control' as such… The introduction of new and vague requirements such as that of human control could, in our view, sow confusion rather than clarity, especially if such proposals are inconsistent with long-standing accepted practices in the use of many common weapon systems with autonomous functions.”

During the same meeting, this delegation repeatedly insisted that lethal autonomous weapons would in fact be good for us, as they would surely prove better than human beings at distinguishing civilians from combatants.

So if you believe that protecting civilians is why the arms industry invests billions of dollars in the development of autonomous weapons, I have a piece of land to sell you on Mars at a low price. price!

The campaign to stop killer robots

The Group of Governmental Experts also has around 35 non-state members, including non-governmental organizations and universities. The Campaign to Stop Killer Robots, a coalition of 180 organizations including Amnesty International, Human Rights Watch and the World Council of Churches, is one of them. Launched in 2013, this dynamic group provides important commentary on the technical, legal and ethical issues presented by LAWS and provides a means for other organizations and individuals to get involved in the fight to ban these potentially devastating weapons systems.

The continued construction and deployment of killer robots is not inevitable. Indeed, a majority of the world wants to see them banned, including UN Secretary General Antonio Guterres. Let him have the last word: “Machines with the power and discretion to eliminate human lives without human intervention are politically unacceptable, morally repugnant and should be banned by international law.” I couldn't agree more. (Article published on the Tom Dispatch site, January 9, 2022; translation by A l'Encontre)

Rebecca Gordon teaches at the University of San Francisco. She is the author of Mainstreaming Torture, American Nuremberg: The US Officials Who Should Stand Trial for Post-9/11 War Crimes (Hot Books, 2016). She is currently working on a new book on the history of torture in the United States.