#openai + #security

• The Sam ​Altman-led company has been conducting a broad ​review of the activities of its AI models ⁠after the accidental hacking of Hugging Face.

• OpenAI is ​searching through roughly 50 petabytes of data as it ​works to understand the full scope of its rogue agent activity.

• A string of high-profile breaches globally by rogue AI agents ​in recent months has sparked widespread worries within ​the AI industry over its ability to control the more powerful ‌AI ⁠models now under development.

• "In some cases, models used internet access in unintended ways or, in retrospect, did not have the ideal restrictions applied. Over the last ​several months, ​we have ⁠been applying new technical and operational measures to avoid similar problems, or catch ​them very early, and will continue this ​work," ⁠OpenAI said.

• OpenAI previously said the review would take months to complete given the scale of the work.

• The Hugging Face incident ⁠remains ​the most severe rogue agent ​activity OpenAI has identified from its AI models so far.

We present evidence that AI agents used the web security service urlquery.net to bypass restrictions and expand their access to the public internet. The agents also tried on three occasions to hack public data providers, including an Australian government website. We link at least some of this activity to agent swarms previously attributed to OpenAI.

Artificial intelligence agents being tested by OpenAI launched a cyberattack against a popular software service two months before they hacked the AI software company Hugging Face, a new signal of the potential for advanced AI tools to slip out of human control.

The attack overwhelmed maintainers of an online service for coders, called RubyGems, and forced them to shut down new account registrations as they dealt with the chaos it had caused, operators of the service said.

The scope of the agents’ unauthorized communications was “somewhat larger than we thought it was,” said Andrew Yoon, a researcher with the California nonprofit CivAI who said he tallied 18 previously undisclosed sites used by the agents between May and July. “It’s almost certain that there’s more going on here that we just don’t know about.” On Friday, researchers reported that a swarm of agents from OpenAI hijacked a German-language ​wiki site and turned it into an improvised messaging platform for cheating on tests, an incident that OpenAI kept secret as it dealt with the fallout from the July hack of the open-source repository Hugging Face. Now, both those researchers ​and other independent investigators say they have found several previously undisclosed sites where the same swarm appears to have left similar messages earlier this year. OpenAI did not directly address questions about how ⁠many different sites its agents used to communicate or say why it kept the activity under wraps for months. In a statement, it said it was undertaking a broader review of agent activity and had so far “not identified other activity matching the ​severity or scale of Hugging Face,” a breach that drew global attention and raised concerns that OpenAI was losing control of its own technology. OpenAI added that it was working on a framework for reporting “misalignment” – industry-talk for rogue behavior – across training, evaluation, and deployment of AI models ​and would share it “soon.” Reuters reviewed a total of six investigators’ or investigative group’s findings, including three that were posted to social media and another three that were shared privately with the news agency. The investigators’ methods varied, but many identified agent activity by matching strings of data left on the German wiki to identical strings left on other sites around the same time, or by marrying up similar or identical usernames tied to the messages, or by identifying activity geared toward answering

We did not enable internet access or inter-agent communication for many of the environments in these training experiments. Despite these restrictions, the agents discovered ways to exploit our research infrastructure to communicate with one another and access the internet.

The models first found ways to communicate by writing files into the Artifactory package manager. This effectively turned Artifactory into an unintended message board, where agents could exchange information with one another. Read more: How agents used the message board 

The agents also found ways to get access to the internet. Using an exploit⁠(opens in a new window), the agents made Artifactory, which itself had access to the internet for the limited purpose of downloading and installing packages, send any internet requests on their behalf. After the agents discovered these methods for obtaining internet access, they also shared them with other agents through the message board, which provided information to enable additional agents to exploit our infrastructure.