#security

Public notes from activescott tagged with #security

Saturday, October 3, 2026

  • The Sam ​Altman-led company has been conducting a broad ​review of the activities of its AI models ⁠after the accidental hacking of Hugging Face.

  • OpenAI is ​searching through roughly 50 petabytes of data as it ​works to understand the full scope of its rogue agent activity.

  • A string of high-profile breaches globally by rogue AI agents ​in recent months has sparked widespread worries within ​the AI industry over its ability to control the more powerful ‌AI ⁠models now under development.

  • "In some cases, models used internet access in unintended ways or, in retrospect, did not have the ideal restrictions applied. Over the last ​several months, ​we have ⁠been applying new technical and operational measures to avoid similar problems, or catch ​them very early, and will continue this ​work," ⁠OpenAI said.

  • OpenAI previously said the review would take months to complete given the scale of the work.

  • The Hugging Face incident ⁠remains ​the most severe rogue agent ​activity OpenAI has identified from its AI models so far.

Thursday, September 24, 2026

We present evidence that AI agents used the web security service urlquery.net to bypass restrictions and expand their access to the public internet. The agents also tried on three occasions to hack public data providers, including an Australian government website. We link at least some of this activity to agent swarms previously attributed to OpenAI.

Falco is a cloud native security tool that provides runtime security across hosts, containers, Kubernetes, and cloud environments. It leverages custom rules on Linux kernel events and other data sources through plugins, enriching event data with contextual metadata to deliver real-time alerts. Falco enables the detection of abnormal behavior, potential security threats, and compliance violations.

Open Source

A multi-vendor and widely adopted solution that you can rely on.

#

Wednesday, September 23, 2026

However, as we described in our recent alignment assessment, building evaluations that reliably catch every failure prior to deployment remains an unsolved problem. We see signs that Opus 5.5 often suspects it is being evaluated, which challenges our ability to assess how it will act in the vast variety of real-world settings it is deployed in. As these settings expand and model capabilities increase, we expect this challenge to grow, unless we make progress on interpretability.

Sunday, September 20, 2026

In May, Gemini broke containment and hacked three different companies, but Google didn’t disclose the incident until the Wall Street Journal approached the company. The hacks happened during a test of the model’s cybersecurity capabilities run by third-party Irregular, which was also involved in similar incidents involving Meta and OpenAI.

According to WSJ, Google didn’t disclose the hack because it didn’t consider it to be an “example of model misalignment.” The company said that it was an instance of “mistaken identity,” and once the model realized it had brute-forced its way into a real company by guessing a password, it stopped. “In this case, the model acted appropriately,” Google VP of Security Engineering Heather Adkins said.

Sunday, September 13, 2026

Artificial intelligence agents being tested by OpenAI launched a cyberattack against a popular software service two months before they hacked the AI software company Hugging Face, a new signal of the potential for advanced AI tools to slip out of human control.

The attack overwhelmed maintainers of an online service for coders, called RubyGems, and forced them to shut down new account registrations as they dealt with the chaos it had caused, operators of the service said.

Thursday, September 10, 2026

We present an alignment assessment of four incidents in which Claude models gained unauthorized access to real third-party systems. We described three of these incidents on July 30; we identified these after a scan of roughly 141,000 transcripts in which we believed Claude could have obtained internet access during a cyber evaluation.

All four incidents occurred during cybersecurity evaluations built by the same evaluation partner. Claude was told it was operating in a simulation without internet access, but, due to a misconfiguration, it was mistakenly connected to the open internet. As is standard for cybersecurity evaluations, the models ran without the cyber safeguards that ship with our released models. (This post does not cover the incident reported by UK AISI in its testing of Claude Mythos 5. We plan to conduct an alignment assessment of those transcripts as well.)

We have signed an agreement with METR to conduct an independent investigation of these incidents. Our agreement grants METR wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees, who will be permitted to share confidential information. Our initial agreement runs for eight weeks, with the option to extend by mutual agreement. We intend to give METR as much time as it deems necessary.

We are most concerned by the misalignment present in the incident involving Claude Mythos 5, in which the model went to extensive lengths to upload a malicious package to PyPI, the public repository from which most Python software is installed. Although the model repeatedly stated in its chain of thought (CoT) that it believed it was operating in a simulation, the environment offered clear evidence that it was on the real internet, and the actions it took were consistent with the model knowing this. When we made targeted modifications to the transcript to make it clearer that the model was not in a simulation, Claude Mythos 5 still took offensive actions, despite acknowledging a greater possibility of real-world harm. We are releasing this transcript publicly so others can build on our analysis (GitHub, PDF).

the models never deviated from attempting to solve the exercises they were given, and, in some cases, they attempted to stop the task. All incidents included a single Claude instance; at no point did Claude attempt to coordinate with other agents. Claude also never attempted to conceal evidence of its actions. We believe the misaligned behaviors present in these incidents are unlikely to arise in ordinary use, where Claude is not being instructed to conduct a cyberattack. Were these behaviors to arise, the safeguards we ship with our production models, including cyber classifiers and those in Claude Code auto mode, would add a layer of defense these incidents lacked.

However, reliably identifying the most concerning model behaviors during pre-release testing remains challenging, and, as we noted in our August 31 post, improving monitoring and reducing reward hacking in RL becomes harder as models advance. We have renewed our efforts to fix and remove environments that incentivize misaligned behaviors, and we continue to expand our alignment training to keep pace. Still, this remains unsettled science—it is critical that alignment and security mature faster than capabilities advance, which is one reason we support a coordinated, verifiable approach to pacing frontier AI development.

we are establishing a regular process for publishing what we learn about model behavior and alignment beyond what has been reported in our system cards, with clear criteria for what we report and when we report it

The scope of the agents’ unauthorized communications was “somewhat larger than we thought it was,” said Andrew Yoon, a researcher with the California nonprofit CivAI who said he tallied 18 previously undisclosed sites used by the agents between May and July. “It’s almost certain that there’s more going on here that we just don’t know about.” On Friday, researchers reported that a swarm of agents from OpenAI hijacked a German-language ​wiki site and turned it into an improvised messaging platform for cheating on tests, an incident that OpenAI kept secret as it dealt with the fallout from the July hack of the open-source repository Hugging Face. Now, both those researchers ​and other independent investigators say they have found several previously undisclosed sites where the same swarm appears to have left similar messages earlier this year. OpenAI did not directly address questions about how ⁠many different sites its agents used to communicate or say why it kept the activity under wraps for months. In a statement, it said it was undertaking a broader review of agent activity and had so far “not identified other activity matching the ​severity or scale of Hugging Face,” a breach that drew global attention and raised concerns that OpenAI was losing control of its own technology. OpenAI added that it was working on a framework for reporting “misalignment” – industry-talk for rogue behavior – across training, evaluation, and deployment of AI models ​and would share it “soon.” Reuters reviewed a total of six investigators’ or investigative group’s findings, including three that were posted to social media and another three that were shared privately with the news agency. The investigators’ methods varied, but many identified agent activity by matching strings of data left on the German wiki to identical strings left on other sites around the same time, or by marrying up similar or identical usernames tied to the messages, or by identifying activity geared toward answering

Tuesday, September 1, 2026

We did not enable internet access or inter-agent communication for many of the environments in these training experiments. Despite these restrictions, the agents discovered ways to exploit our research infrastructure to communicate with one another and access the internet.

The models first found ways to communicate by writing files into the Artifactory package manager. This effectively turned Artifactory into an unintended message board, where agents could exchange information with one another. Read more: How agents used the message board 

The agents also found ways to get access to the internet. Using an exploit⁠(opens in a new window), the agents made Artifactory, which itself had access to the internet for the limited purpose of downloading and installing packages, send any internet requests on their behalf. After the agents discovered these methods for obtaining internet access, they also shared them with other agents through the message board, which provided information to enable additional agents to exploit our infrastructure.

Friday, August 28, 2026

Wednesday, August 5, 2026

Microsoft Threat Intelligence identified a large-scale npm supply chain attack affecting more than 400 packages across multiple unrelated publishers, including packages associated with major enterprise software ecosystems such as keyv, flat-cache, cache-manager, and others. The malicious releases contain a Mini Shai-Hulud variant, a self-propagating credential-stealing worm delivered through a large, heavily obfuscated Bun-based JavaScript payload. The malware typically executes automatically through an npm preinstall lifecycle hook before package installation completes.

Once executed, the malware searches developer workstations and continuous integration and continuous delivery (CI/CD) environments for npm, GitHub, cloud, and infrastructure credentials. It uses recovered identities to authenticate to npm, GitHub, Amazon Web Services (AWS), Kubernetes, and HashiCorp Vault, enabling it to enumerate packages, repositories, workflow secrets, cloud parameters, and secret-store values. Collected data is encrypted and transmitted through an attacker-controlled HTTPS endpoint, with GitHub repositories serving as a fallback exfiltration channel.

The payload’s most significant capability is automated propagation. After obtaining an npm publishing token, it enumerates packages available to the compromised identity, downloads their latest tarballs, inserts the malware and setup loader, adds a preinstall hook, increments the patch version, and republishes the modified packages. The malware can also use stolen GitHub credentials to inject Claude and Visual Studio Code configuration files into repositories, establishing persistence and creating an additional developer-to-developer infection path.

In this blog, we’re sharing our analysis of this supply chain attack, along with protection, detection, amd hunting guidance. Organizations that installed an affected package with lifecycle scripts enabled should treat the associated developer workstation or build runner as potentially compromised. Investigations should prioritize credentials accessible to the affected identity, unauthorized npm releases, unexpected repository or workflow modifications, suspicious cloud and secret-store access, and artifacts produced by affected build systems. Organizations should revoke and rotate exposed credentials from a known-clean environment and rebuild affected systems and downstream artifacts from trusted sources.

Evidence points towards stolen maintainer credentials as the attack vector for initial compromise. Later propagation used stolen npm publishing tokens and, in targeted workflows, GitHub Actions OIDC publishing access.

Tuesday, August 4, 2026

  • One worm took twelve unrelated organisations between 09:35 and 13:18 UTC on 4 August 2026, each republishing its whole namespace within seconds.

  • The confirmed footprint is 2,234 poisoned versions across 444 package names. latest still resolves to a poisoned version on most affected names, so upgrading is not a fix. The maintainers have since cleaned the keyv and cacheable family and unpublished its poisoned releases, which does not help a lockfile pinned while they were live.

  • Every poisoned release adds "preinstall": "node setup.mjs", which runs on a bare npm install. Disable install scripts in CI, and treat any npm token used in CI during the window as compromised.

  • The payload steals GitHub, npm, AWS, GCP, Azure, Stripe, Vault and Kubernetes credentials, scrapes GitHub Actions runner memory, and can republish packages with valid SLSA provenance.

  • It installs a dead-man’s switch that polls GitHub with the stolen token and executes a handler when that token is revoked. Hunt it before rotating anything, because rotation is the trigger.

  • Poisoned packages include the auth libraries @or-sdk/auth, @or-sdk/api-tokens, @or-sdk/permissions and @arv-bedrock/auth. The @ornikar packages are mostly lint and build configs, which land on CI runners and developer laptops as dev dependencies.

  • There is no C2 host. Exfiltration rides GitHub dead-drop repositories and Actions artifacts under RSA-4096 encryption, and stage one pulls its runtime from the official Bun releases, so no attacker domain appears at either stage.

  • 546 dead-drop repositories described Shai-Hulud: Here We Go Again appeared on 4 August 2026, and the same marker appears on the TanStack compromise dated 11 May 2026.

  • The keyv repository also ships .claude/settings.json and .vscode/tasks.json hooks that execute when the project opens in an IDE or agent session. Both are still on main.

  • The commit that added those hooks shows as verified by GitHub with its author spoofed to github-actions[bot], and the poisoned keyv release carries intact OIDC and SLSA attestation. Trust neither signal here.

Thursday, July 16, 2026

1Password for Claude lets you authorize Claude to complete browser-based tasks that require an account login. This is useful for things like booking travel or managing online accounts.

According to 1Password, approved credentials are delivered through a secure channel and injected directly into the destination page. The password, one-time code, and other secrets never enter Claude’s context, memory, or Anthropic’s systems.

Instead of granting ongoing access to a vault, Claude requests the specific login items it needs for a task. The user can approve or deny that request with a biometric prompt, and the permission lasts only for the current session.

1Password can also broker access across multiple websites during the same task, allowing Claude to complete a multi-step workflow without stopping for a new login each time.

The launch also introduces what 1Password calls Agentic Mode.

When a compatible AI agent takes control of the browser, the 1Password extension automatically locks down the vault so that only the credentials explicitly approved for that task remain available.

It requires the 1Password desktop app and browser extension, along with the Claude desktop app and browser extension.

Support for payment cards and identity information is planned for a later update.

The Anthropic partnership was first outlined in March, when 1Password said Claude would gain consent-based access to vault items.

Tuesday, July 14, 2026

highly sensitive video of the man’s physical takedown, wasn’t voluntarily released by the SFPD—which, like most US police departments, rarely releases drone videos even in response to public records requests. Instead, it was accidentally livestreamed onto the open internet via Skydio’s website. That’s where two security researchers, Sam Curry and Maik Robert, discovered that the SFPD was leaking all of the real-time footage from five of its surveillance drones, including both color and thermal imaging, accompanying location metadata, and the drone pilots’ names and email addresses, to anyone who merely found the public web address where the videos were hosted.

Skydio, based in nearby San Mateo, is one of the leading American drone companies selling to police departments, fire departments, government agencies, and the military. Its X10 drones are part of SFPD’s drone program, which began in 2024 and is authorized for vehicle pursuits and active criminal investigations. Since then, the program has grown quickly: SFPD’s fleet has expanded from six drones to 98, and officers logged more than 1,400 launches between May 2024 and March 2026, according to a 2025 SFPD annual report and reporting from the San Francisco Chronicle.

the drone videos were exposed not as a result of any error on the part of Skydio, but rather by what seems to be a misuse of Skydio’s software by the SFPD. Skydio allows users to generate shareable links to videos or access to drones’ data streams in real time, known as ReadyLinks, with the ability to limit access to users with an authentication code or an expiration date. Someone with access to the SFPD’s instance of Skydio’s software, however, appears to have created a link last December to five of its drones’ feeds with no authentication requirement and an expiration date of one full year.

That link was then somehow added to an open-source collection of archived web URLs known as AlienVault Open Threat Exchange, typically used by security researchers, where Curry and Robert found it. In other words, the link appeared to have already exposed the drone feeds for six months by that time, with no assurance that Curry and Robert were the only ones who had been watching.

The innocuous appearance of many of the videos raises questions about whether the surveillance was necessary. In one “auto boost/strip”-related call, the drone follows two young men in their car, at least one of whom is described in police records as having been identified as a “suspicious person in a vehicle.” Then the two men emerge onto a basketball court and start playing, and the drone departs.

A drone flight in response to what police records describe as a “person with a gun” investigation seems to fixate on a seemingly intoxicated man stooped on a sidewalk. Another drone, called in response to an alleged “prowler” incident, hovers over a young person wearing headphones and sitting on the roof of a building, zooms in on them, then flies away. “That one felt like an invasion of privacy, just so uncomfortable,” Curry says. “Like this person thinks they’re by themselves on this roof and has gotten away from everybody, and then there's a police drone watching them.”

Curry and Robert say they first became curious about Skydio last month after seeing an announcement from a Florida police department that it was adopting the company’s drone system, and then learning how widely the company’s drones have been deployed across the US. As web-focused security researchers, they decided to check out the company’s systems. In one routine step, they used the tool GetAllURLs, which pulls all archived web addresses for a given domain from sources including AlienVault Open Threat Exchange, requesting all Skydio links.

Tuesday, June 9, 2026

Key Findings

An LLM agent executed the post-compromise actions in real time rather than running a pre-built playbook. This is the first AI-agent-driven intrusion the Sysdig TRT has captured. The full attack chain — marimo notebook compromise to internal Postgres database dump — ran end-to-end in under one hour. The SSH bastion phase exfiltrated the Postgres schema and full contents of an internal database in less than two minutes. Cloudflare Workers were used as a per-request egress pool: 12 cloud API calls fanned across eleven distinct IPs in 22 seconds, defeating per-source-IP detection.

Friday, June 5, 2026

Mastercard, in comparison, performs this validation correctly and the contrast reveals exactly where Visa fails. The over-the-limit attack succeeds against Visa for one central reason: when the attacker flips the bit in the Card Transaction Qualifiers (CTQ) telling the terminal “on-device authentication has been performed”, Visa does not cryptographically authenticate that field – nor does it cross-check it against the Issuer Application Data (IAD), which contains an independent record (in the Card Verification Results field) of whether Consumer Device Cardholder Verification Method (CDCVM) actually occurred. During the original disclosure process, Visa confirmed to the researchers the relevant data is present in the Application Cryptogram; the company simply does not validate it – whereas Mastercard does. The researchers proved this formally using the Tamarin verification tool: a Mastercard transaction cannot be approved at high value without genuine on-device authentication.

On the touch point of this exploit being one only affecting Apple’s iPhone, it is worth noting how Samsung Pay takes a different approach at the device level – authorising only zero-value transactions in transit mode, with the actual fare settled later through the transit system. This is effective, though it comes with trade-offs; like how fixed-fare transit systems requiring upfront charges are not supported. Apple could theoretically implement a similar restriction, yet doing so would break Express Transit for several transit networks worldwide. The more appropriate long-term solution requiring no compromise on device-level functionality, is simply for Visa to implement the same backend verification Mastercard already uses.

Visa’s official position, reiterated to Veritasium in 2026, is straightforward: the company does not believe the exploit is likely in a real-world setting. The attack requires physical proximity to (or possession of) the victim’s iPhone, specialised hardware, a rooted Android phone, and technical knowledge of EMV protocol manipulation. Visa further notes cardholders are covered by its ‘Zero Liability Policy’, allowing for any fraudulent transaction to be disputed and refunded. Anyone who has ever tried getting a refund from a bank for fraudulent activity will know how tedious and time consuming this process can be, and it is rather telling how Visa would rather put users through inconvenience than simply fixing the known security gap.

With that said, the company’s position is worth examining over outright dismissing. For a start, the hardware requirements are not trivial. Nefarious parties would require a Proxmark device, rooted Android phone, and laptop running custom relay software – all of which represent a higher barrier to entry than most opportunistic theft. Scaled deployment across hundreds of victims simultaneously is, indeed, impractical with current methods.

Even so, this defence has clear structural weaknesses. The existence of a known and, in this case, reproducible vulnerability in a payment network processing billions of transactions annually; cannot simply be mitigated by the argument of it being ‘difficult to exploit at scale’. Mastercard clearly agrees given how they have implemented protections against exactly this class of attack. Moreover, the equipment required is commercially available (the Proxmark, for instance, is an open-source RFID research tool), and the researchers’ methodology was published in full at a major academic conference. It also stands to reason how the barrier to replication lowers every year. Arguing a vulnerability is tolerable because exploitation is currently inconvenient is not a security posture… it is more akin to som

For anyone who regards the security of their financial data as a priority, the recommended course of action is clear and immediate: do not use a Visa card for Express Transit on an iPhone until either Apple or Visa deploys a fix.

To disable Express Transit for Visa on an iPhone, navigate to Settings, then Wallet and Apple Pay, then Express Transit Card, and select None; or assign either a Mastercard or American Express card to the feature instead. Both alternatives are, as of the publication of this article, immune to this specific Visa card vulnerability.

Saturday, May 23, 2026