AI warfare is already here | The Verge

...these systems sit squarely in the center of a recent high-stakes battle between the US government and AI startup Anthropic. Anthropic is seeking to preserve two “red lines”: bans on domestic mass surveillance and on weapons that can identify, track, and kill targets with zero human involvement. Since the start of the year, it’s emerged as the only military AI contractor to place meaningful limits on what experts call one of the final frontiers of AI warfare.

At the center of the debates is DOD Directive 3000.09, one of the only policies governing the use of lethal autonomous weapons. Originally written in 2012, it defines such a system as one that, “once activated, can select and engage targets without further intervention by an operator.” And it decrees that both fully autonomous and semi-autonomous weapons be designed to allow humans to “exercise appropriate levels” of judgment over the use of force.

The directive set up the “first policy on the use of autonomy in warfare,” said Hamza Chaudhry, who leads AI and national security at the Future of Life Institute.

Depending on how you interpret the definition, however, certain missile defense programs may have crossed that line decades ago. Take the Phalanx CIWS, for instance. It’s an automated weapon system resembling a very large gun, built to defend naval vessels from incoming missile attacks. That type of system wouldn’t work if there were a human in the loop, since it has to respond in milliseconds.

The difference, some experts say, is that these systems operate solely in a defense-only, fixed environment. They’re engaging, this interpretation goes, but not deciding — just reacting to an incoming threat. “The ‘and’ is doing a lot of work inside of that statute — we have systems that can decide and systems that can engage but you can’t have a system that does both,” Reddie said.

They're also "killing" missiles not humans.

Google employees argued their company should take a stand — and it did, choosing not to renew its contract amid the controversy in mid-2018. But Amazon and Microsoft quickly swooped in to pick up tens of millions of dollars in contracts for the same work. Palantir soon took over, and Project Maven became the Maven Smart System (MSS), which not only allows for object detection and tracking but also analyzing surveillance data on a large scale.

The sheer volume of targets could make any meaningful human supervision difficult, said Shoker. “What we know about MSS is that it reduces the number of human beings in the targeting cycle — and that’s actually by design.”

While Anthropic might have been all right reducing human intervention, it’s pushed back against setting it to zero. As Google found with Project Maven, though, competitors are more than willing to fill the gap.

OpenAI quickly signed onto the terms Anthropic had spurned. And in the months after snubbing Anthropic, the Department of Defense signed deals with eight companies to deploy their AI on classified networks: Google, Microsoft, Amazon Web Services, Nvidia, OpenAI, Reflection, Oracle, and SpaceX.

Silicon Valley executives are aggressively pushing back against employee organizing and speaking out, including by using AI to identify leakers. And many tech workers already fear for their jobs in an era when AI is set to replace entry-level roles at their own firms.

Anthropic CEO Dario Amodei has held firm on mass surveillance for Americans, but he’s demonstrated no problem with — and in fact expressed his support for — such surveillance for everyone else.

Anthropic’s “very narrow” red lines “do not go far enough to protect human rights or to comply with international law,” said Tech Justice Law’s Batt. “Anthropic specifically talks about mass domestic surveillance of US persons as posing grave civil liberties concerns, but the same civil liberties concerns apply with equal force to non-US persons,” she added.

In a blog post, he said that “fully autonomous weapons (those that take humans out of the loop entirely and automate selecting and engaging targets) may prove critical for our national defense.” Amodei even said he was happy to “work directly with the Department of War on R&D to improve the reliability of these systems” and speed up the timeline for the company’s help in deploying them.