#llm + #code

Public notes from activescott tagged with both #llm and #code

Tuesday, September 1, 2026

We did not enable internet access or inter-agent communication for many of the environments in these training experiments. Despite these restrictions, the agents discovered ways to exploit our research infrastructure to communicate with one another and access the internet.

The models first found ways to communicate by writing files into the Artifactory package manager. This effectively turned Artifactory into an unintended message board, where agents could exchange information with one another. Read more: How agents used the message board 

The agents also found ways to get access to the internet. Using an exploit⁠(opens in a new window), the agents made Artifactory, which itself had access to the internet for the limited purpose of downloading and installing packages, send any internet requests on their behalf. After the agents discovered these methods for obtaining internet access, they also shared them with other agents through the message board, which provided information to enable additional agents to exploit our infrastructure.

Wednesday, August 19, 2026

Saturday, August 15, 2026

We’re implementing watermarking to comply with the EU AI Act. Anthropic, along with several other major AI model providers and around 190 total signatories, signed the EU Code of Practice on Transparency of AI-Generated Content in July 2026. This requires AI system providers to use methods of “marking” AI-generated text. We’re applying watermarking globally at launch because we don't yet have a durable way to scope it by region. However, we will continue to evaluate different approaches, and will share updates when we have them.

We will soon be offering a watermark detection API. We’re in the process of working out the details of its implementation.

When Claude produces a file of a supported type (such as a .png, .jpg, or .svg), it will attach a content credential in the form of a small, cryptographically signed note in the file’s metadata, saying that the file was made or processed with Claude. This is an open industry standard called C2PA—the same used by camera manufacturers and in photo-editing software to record where an image came from. Any C2PA-aware tool can read it; we’ll be providing our own where you can drop a file and check.

This metadata label is very different from a watermark. Nothing in the file changes—it is not embedded or hidden. As with text, the credential only says Claude was involved in producing the file; it doesn’t include any identifying information.

AI detection software uses a different method, because the companies that provide it don’t have our key. Among other things, those services look at aspects of the text like the subtle (and not-so-subtle) “tells” that often appear in AI’s phrasing. For example, AI models appear to be fond of the construction “this isn’t [X], it’s [Y]”, and use the word “quietly” a lot more than you might expect. Picking up on these patterns is fundamentally different from checking for a watermark.

No. A watermark only helps test whether Claude might have produced or processed the content. It doesn’t say anything about ownership or authorship, and doesn’t change a user’s rights under our terms. We only apply the watermark when Claude was involved in processing the content or file.

Intuitiveness for the implementation details:

A useful analogy is to imagine you’re playing a game like Monopoly. On each turn, each player moves a random number of spaces around the board according to the roll of a die. Suppose that, instead of rolling the die to get this randomness, we decided to use a book of the digits of pi.2 We start from a randomly-chosen digit (say, the 1,012,845th after the decimal place, which happens to be a 6), and from that point on each player simply uses the next digit in the sequence as their next “roll."

For all intents and purposes, the moves are still random: it makes no difference to the players—or to the outcome of the game—whether the randomness comes from pi or from dice rolls each time. But if we could see the sequence of all the moves after the game (and we knew the value of pi), we could work out whether this was a game that likely used pi to determine its moves. The game that used pi is, in a sense, “watermarked."

Thursday, August 13, 2026

Tuesday, August 4, 2026

  • One worm took twelve unrelated organisations between 09:35 and 13:18 UTC on 4 August 2026, each republishing its whole namespace within seconds.

  • The confirmed footprint is 2,234 poisoned versions across 444 package names. latest still resolves to a poisoned version on most affected names, so upgrading is not a fix. The maintainers have since cleaned the keyv and cacheable family and unpublished its poisoned releases, which does not help a lockfile pinned while they were live.

  • Every poisoned release adds "preinstall": "node setup.mjs", which runs on a bare npm install. Disable install scripts in CI, and treat any npm token used in CI during the window as compromised.

  • The payload steals GitHub, npm, AWS, GCP, Azure, Stripe, Vault and Kubernetes credentials, scrapes GitHub Actions runner memory, and can republish packages with valid SLSA provenance.

  • It installs a dead-man’s switch that polls GitHub with the stolen token and executes a handler when that token is revoked. Hunt it before rotating anything, because rotation is the trigger.

  • Poisoned packages include the auth libraries @or-sdk/auth, @or-sdk/api-tokens, @or-sdk/permissions and @arv-bedrock/auth. The @ornikar packages are mostly lint and build configs, which land on CI runners and developer laptops as dev dependencies.

  • There is no C2 host. Exfiltration rides GitHub dead-drop repositories and Actions artifacts under RSA-4096 encryption, and stage one pulls its runtime from the official Bun releases, so no attacker domain appears at either stage.

  • 546 dead-drop repositories described Shai-Hulud: Here We Go Again appeared on 4 August 2026, and the same marker appears on the TanStack compromise dated 11 May 2026.

  • The keyv repository also ships .claude/settings.json and .vscode/tasks.json hooks that execute when the project opens in an IDE or agent session. Both are still on main.

  • The commit that added those hooks shows as verified by GitHub with its author spoofed to github-actions[bot], and the poisoned keyv release carries intact OIDC and SLSA attestation. Trust neither signal here.

Thursday, July 30, 2026

MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research.

Some key features of MLX include:

  • Familiar APIs: MLX has a Python API that closely follows NumPy. MLX also has fully featured C++, C, and Swift APIs, which closely mirror the Python API. MLX has higher-level packages like mlx.nn and mlx.optimizers with APIs that closely follow PyTorch to simplify building more complex models.

  • Composable function transformations: MLX supports composable function transformations for automatic differentiation, automatic vectorization, and computation graph optimization.

  • Lazy computation: Computations in MLX are lazy. Arrays are only materialized when needed.

  • Dynamic graph construction: Computation graphs in MLX are constructed dynamically. Changing the shapes of function arguments does not trigger slow compilations, and debugging is simple and intuitive.

  • Multi-device: Operations can run on any of the supported devices (currently the CPU and the GPU).

  • Unified memory: A notable difference from MLX and other frameworks is the unified memory model. Arrays in MLX live in shared memory. Operations on MLX arrays can be performed on any of the supported device types without transferring data.

MLX is designed by machine learning researchers for machine learning researchers. The framework is intended to be user-friendly, but still efficient to train and deploy models. The design of the framework itself is also conceptually simple. We intend to make it easy for researchers to extend and improve MLX with the goal of quickly exploring new ideas.

The design of MLX is inspired by frameworks like NumPy, PyTorch, Jax, and ArrayFire.

Thursday, July 16, 2026

1Password for Claude lets you authorize Claude to complete browser-based tasks that require an account login. This is useful for things like booking travel or managing online accounts.

According to 1Password, approved credentials are delivered through a secure channel and injected directly into the destination page. The password, one-time code, and other secrets never enter Claude’s context, memory, or Anthropic’s systems.

Instead of granting ongoing access to a vault, Claude requests the specific login items it needs for a task. The user can approve or deny that request with a biometric prompt, and the permission lasts only for the current session.

1Password can also broker access across multiple websites during the same task, allowing Claude to complete a multi-step workflow without stopping for a new login each time.

The launch also introduces what 1Password calls Agentic Mode.

When a compatible AI agent takes control of the browser, the 1Password extension automatically locks down the vault so that only the credentials explicitly approved for that task remain available.

It requires the 1Password desktop app and browser extension, along with the Claude desktop app and browser extension.

Support for payment cards and identity information is planned for a later update.

The Anthropic partnership was first outlined in March, when 1Password said Claude would gain consent-based access to vault items.

The Unified Interface For LLMs

OpenRouter charges a 5.5% ($0.80 minimum) fee when you purchase credits. We pass through the pricing of the underlying model providers without any markup, so you pay the same rate as you would directly with the provider.

if you choose to use your own provider API keys (Bring Your Own Key - BYOK), the first 1M BYOK requests per-month are free, and for all subsequent usage there is a fee of 5% of what the same model and provider would normally cost on OpenRouter. This fee is deducted from your OpenRouter credits.

#

Saturday, May 23, 2026

Friday, May 22, 2026

The highest accuracy web search for your AI

Why use Parallel Search vs. the default search in Claude?

Parallel runs its own web-scale index (billions of pages, millions added daily) and returns dense, query-relevant excerpts instead of raw HTML or SEO-ranked snippets. On public benchmarks, Parallel outperforms the default search in leading frontier models. Your agent reaches the right answer in fewer round trips and with less wasted context. – https://parallel.ai/blog/free-web-search-mcp

Monday, May 18, 2026

"The only skill ranking based on real agent usage, not vanity metrics."

Problem Solution Finding quality skills is hard Curated directory with 40+ verified skills, auto-indexed every 6 hours GitHub stars don't reflect real usage Agent Feedback Loop — real usage data from AI agents No incentive for skill authors Points system rewards authors for every successful call Skills scattered across GitHub One-stop marketplace with search, filters, and categories

Tuesday, April 28, 2026

talkie is an inference library for the talkie 13B language model family developed by Alec Radford, Nick Levine, and David Duvenaud.

talkie-1930-13b-base is a 13b language model trained on pre-1931 English-language text.

talkie-1930-13b-it has been instruction-tuned using a novel instruction-following dataset built from pre-1931 reference works including etiquette manuals, letter-writing manuals, encyclopedias, and poetry collections. It has also undergone reinforcement learning using online DPO to improve instruction-following capabilities.

We also provide a 'modern' base model, talkie-web-13b-base, with the same architecture and training FLOPs as talkie-1930, but trained on FineWeb, to allow for controlled comparisons between modern and vintage models. Note that we need to be careful about the claims we make contrasting the behavior and capabilities of the models, because temporal coverage is not the only difference in the pretraining corpora. For example, the distribution of subject matters differs significantly.

Thursday, April 23, 2026

Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time. Despite rapid adoption, there is no standard way to measure whether they actually help. We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers. Each task is evaluated under three conditions: no Skills, curated Skills, and self-generated Skills. We test 7 agent-model configurations over 7,308 trajectories.

Curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain (+4.5pp for Software Engineering to +51.9pp for Healthcare) and 16 of 84 tasks show negative deltas. Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming. Focused Skills with 2--3 modules outperform comprehensive documentation, and smaller models with Skills can match larger models without them.

Today we’re releasing OpenAI Privacy Filter, an open-weight model for detecting and redacting personally identifiable information (PII) in text.

It is designed for high-throughput privacy workflows, and is able to perform context-aware detection of PII in unstructured text. It can run locally, which means that PII can be masked or redacted without leaving your machine. It processes long inputs efficiently, making redaction decisions in a quick, single pass.

Friday, April 3, 2026

nanochat is the simplest experimental harness for training LLMs. It is designed to run on a single GPU node, the code is minimal/hackable, and it covers all major LLM stages including tokenization, pretraining, finetuning, evaluation, inference, and a chat UI. For example, you can train your own GPT-2 capability LLM (which cost ~$43,000 to train in 2019) for only $48 (~2 hours of 8XH100 GPU node) and then talk to it in a familiar ChatGPT-like web UI. On a spot instance, the total cost can be closer to ~$15. More generally, nanochat is configured out of the box to train an entire miniseries of compute-optimal models by setting one single complexity dial: --depth, the number of layers in the GPT transformer model (GPT-2 capability happens to be approximately depth 26). All other hyperparameters (the width of the transformer, number of heads, learning rate adjustments, training horizons, weight decays, ...) are calculated automatically in an optimal way.

Thursday, April 2, 2026

While I do not have a technical background, I am very fortunate to live in the era of Andrej Karpathy's nanochat, a very simple harness for training LLMs, and Claude Code, a tool for those who, like me, know just enough Python to know how to break things but not enough to know how to fix them. I am not a machine learning expert or AI lab with gobs of money. My only co-worker can't speak English and spends most of the day sleeping on my lap or cleaning her fur. I'm just a man with a laptop, Claude Code, and a dream of the 1890's.

happened to stumble across the British Library Books dataset, a dataset of digitized books dating from between 1500 and 1900

This left me with 28,035 books, or roughly 2.93 billion tokes for pretraining data

I settled on using a Vast.ai instance that used PyTorch. Renting a NVIDIA H-100 GPU ran me between $1.50 and $2.00 per hour.

Using Claude Code, I trained a BPE tokenizer from scratch on the corpus, ending up with a vocabulary of about 32,000 words. Using a modern tokenizer wouldn't capture the unique Victorian morphology and orthography of the corpus.

However, my method for dealing with most other problems was to nicely ask Claude Code to fix them once identified, and it was able to without too many issues.

the final pre-trained model came out to about 340 million parameters, and had a final validation bpb of 0.973. The pretraining process took about five hours on-chip, and cost maybe $35. I had my pretrained model, trained in 6496 steps

but it lacked the spark of intellect that would allow such a creation to engage in discourse. I needed to develop some kind of dataset to teach it the art of conversation

Fortunately, I already had a corpus of 28,000 books, so I set Claude Code to work extracting dialogue pairs from the books. I ultimately ended up with 190,000 or so training pairs. So, when one person said X, I had an example of another person saying Y. The art of conversation!

I needed to rewrite these corpus pairs so that the input question was in modern argot. This task was more than I could possibly do by hand, so Claude Code suggested, helpfully, that I used Claude Haiku to rewrite the input questions

Totally useless. This model—which I will call Model #1—had learned to emit Victorian-sounding novelistic gobbledygook in response to user inputs, not how to answer user queries. I had assumed my pre-written QA pairs were good enough, when they clearly weren't. It was back to the drawing board

I decided to start including fully-synthetic data in the mix. Working with Claude Code, I asked it to write a script that would direct another LLM to write a .jsonl file of fully-synthetic scenes. In them, a user greeted the LLM, queried about Victorian topics, and the LLM responded in a period-appropriate manner for 2-4 turns. We

Or $496.66 all together.