autotraining models with markdown

The idea: give an AI agent a small but real LLM training setup and let it experiment autonomously overnight. It modifies the code, trains for 5 minutes, checks if the result improved, keeps or discards, and repeats. You wake up in the morning to a log of experiments and (hopefully) a better model. The training code here is a simplified single-GPU implementation of nanochat. The core idea is that you're not touching any of the Python files like you normally would as a researcher. Instead, you are programming the program.md Markdown files that provide context to the AI agents and set up your autonomous research org. The default program.md in this repo is intentionally kept as a bare bones baseline, though it's obvious how one would iterate on it over time to find the "research org code" that achieves the fastest research progress, how you'd add more agents to the mix, etc. A bit more context on this project is here in this tweet.

We did the math. At $185 billion a year, in eight years, Google would be spending $1.5 trillion, slightly more than OpenAI has committed to spend over the same time period. Extend that out to 10 years, as Vahdat noted, and Google would be spending $1.9 trillion.

Vahdat is clear that this is “not a promise” that Google would spend that much over the next 10 years. But the decade-long view he takes suggests the scope of Google’s bet. “The point here is that we are, at Google, investing at the highest levels,” he says.

There’s a big difference between Google’s data center ambitions and OpenAI’s: Google is a money-making machine. In the fourth quarter, Google parent Alphabet raked in $113 billion in revenue; for the full year, sales topped $400 billion for the first time in the company’s more than 25 year history. By comparison, OpenAI is spending at similar levels and only brought in about $13 billion in revenue last year — a tiny fraction of Google’s revenue, and less than half of Google’s cash reserves.

Google’s TPUs previously were only used in house for Google’s own infrastructure — to power consumer apps like Gmail and YouTube, and eventually train self-driving cars and develop and run AI models like Gemini. Now, they’re one of the industry’s go-tos: maybe not as popular as Nvidia’s top of the line Blackwells, but still useful for pretraining and operating AI models at scale. Google first started selling access to them through a cloud service in 2018, letting other companies rent out processing power. But more recently, Google has inked high profile deals, like a big contract with Anthropic, and has reportedly been in talks with Meta to use its chips. In December, Morgan Stanley estimated that TPUs could generate $13 billion for Google by 2027. “It is fair to say that the demand for cloud TPUs has been unprecedented,” Vahdat says, particularly in the last few years.

In August, Vahdat, Google Chief Scientist Jeff Dean, and 10 other researchers and execs at the company, co-published a paper aiming to contextualize AI’s power guzzling. The paper says that the median prompt for Google’s Gemini AI model uses the same amount of energy it takes to power 9 seconds of television and consumes around five drops of water, which they write is “substantially lower than many public estimates.” (One report says large data centers can consume up to 5 million gallons per day, equivalent to the water use of a town populated by up to 50,000 people.)

Coding After Coders: Summary

The New Reality of AI-Assisted Programming

• Elite software developers now rarely write code themselves — instead, they direct AI agents in plain English
• Tools like Claude Code deploy multiple agents simultaneously: one writes, one tests, one supervises
• Tasks that once took days now take under an hour

The Strange New Workflow

• Developers spend their days describing intent to AI, reviewing the AI's "plan," then letting agents execute
• When agents misbehave, developers have resorted to scolding, pleading, ALL-CAPS commands, and emotionally charged language ("embarrassing," "national security imperative") — and it seems to work
• Prompt files have become records of hard-won rules to constrain unpredictable AI behavior

Economic Stakes

• Coding was once considered near-guaranteed, high-paying employment ($200K+)
• It may be the first expensive white-collar skill AI can fully replace — unlike AI video or legal briefs, AI-generated code that passes tests is indistinguishable in value from human-written code
• Irony noted: Silicon Valley workers, who told others to "learn to code," got automated first

Developer Sentiment: Mostly Euphoric

• Most developers interviewed were energized, not demoralized — reporting 10x to 100x productivity gains
• Key insight from tech executive Anil Dash: unlike creative fields where AI removes the soulful work and leaves drudgery, in coding AI removes the drudgery and leaves the soulful parts

Historical Context: A Long Arc of Abstraction

• Each programming era simplified the one before: Assembly → high-level languages (Python) → open-source packages → now natural language intent
• AI represents the highest abstraction layer yet: developers no longer need to manage syntax, memory, or debugging minutiae
• The open question, now being asked at Anthropic itself: what is coding, fundamentally, when the code-writing is gone?

It does, kinda, matter that Hegseth turned a simple contract dispute into an attempted corporate death sentence, weaponizing a supply-chain security designation that was clearly designed for tech the US government fears could be infiltrated by hostile foreign nations.

Yet, under Hegseth’s order, Chinese AI models would technically be more welcome in America’s military supply chain than Anthropic’s. The “supply chain risk” designation is now being used to punish a domestic company for having safety guidelines. DeepSeek, with its direct ties to the Chinese government, faces fewer restrictions than a San Francisco company that committed the cardinal sin of asking for human oversight on killing decisions.

One source familiar with the Pentagon’s negotiations with AI companies confirmed that OpenAI’s deal is much softer than the one Anthropic was pushing for, thanks largely to three words: “any lawful use.” In negotiations, the person said, the Pentagon wouldn’t back down on its desire to collect and analyze bulk data on Americans. If you look line-by-line at the OpenAI terms, the source said, every aspect of it boils down to: If it’s technically legal, then the US military can use OpenAI’s technology to carry it out. And over the past decades, the US government has stretched the definition of “technically legal” to cover sweeping mass surveillance programs — and more.

In the years after 9/11, US intelligence agencies ramped up a surveillance system that they determined fell within the legal limits OpenAI cites, including multiple mass domestic spying operations (along with apparently highly invasive international ones). In 2013, National Security Agency intelligence contractor Edward Snowden revealed the extent of some of these programs, such as reportedly collecting telephone records of Verizon customers on an “ongoing, daily” basis, and gathering bulk data on individuals from tech companies like Microsoft, Google, and Apple via a secretive program called PRISM. Despite promises of reform from intelligence agencies and attempts at legal changes, few significant limits to these powers were enacted. Mike Masnick, founder of Techdirt, said online that OpenAI’s deal “absolutely does allow for domestic surveillance. EO 12333 is how the NSA hides its domestic surveillance by capturing communications by tapping into lines outside the US even if it contains info from/on US persons.”

The danger here isn’t just about one contract; it’s about the precedent. If the Pentagon successfully bullies Anthropic into submission or replaces it with a more “flexible” competitor, we are effectively witnessing the birth of an intentionally unethical AI.

The Death of Human Agency When AI is integrated into weaponry for “all lawful purposes” without restrictions on autonomy, we invite the Responsibility Gap. If an AI-driven drone swarm misidentifies a target, who is at fault? By removing the “human-in-the-loop” requirement, the military is seeking a weapon that offers the ultimate prize of war: lethality without accountability. Surveillance as a Service Existing U.S. laws were written for wiretaps, not for generative AI that can ingest millions of data points to build predictive profiles. Under an “all lawful purposes” mandate, an LLM could be turned into a digital Panopticon. Anthropic has warned that current laws have not caught up to what AI can do in terms of analyzing open-source intelligence on citizens. The Moral Race to the Bottom If the Pentagon blacklists Anthropic, it sends a clear message to competitors: Safety is a liability. To win government billions, firms will be incentivized to strip away safety layers. Reports already suggest OpenAI, Google, and xAI have shown more “flexibility” regarding the Pentagon’s demands.

The Pentagon’s “supply chain threat” maneuver is a scorched-earth tactic designed to force Silicon Valley to choose between its values and its bottom line.

If Anthropic stands firm, it may lose $200 million in revenue and a seat at the defense table. But if they cave, they may well be providing the operating system for the very “Terminator” future they were founded to prevent. In the world of 2026, the most dangerous threat to the supply chain might just be an AI that has been ordered to stop caring about ethics.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

It’s clear that the huge spending on AI is adding to the U.S. economy, but the available economic data doesn’t neatly capture its effects. The debating economists and the slippery data suggest that if the technology does start to reshape the economy, it may be challenging to detect and clearly measure. That may leave political and corporate leaders to choose the numbers that fit their preferred narratives on how AI is changing American life and work.

That’s because the $31 trillion in yearly U.S. gross domestic product, the widest measure of the economy, tallies only the final value of products and services produced domestically. Spending on imports and foreign made components is subtracted because it boosts the economies of other countries, not that of the United States.

Roughly three-quarters of the cost of an AI data center is for the computer gear and parts such as computer chips that go inside of it, technology analysts estimate. America’s AI champions, including the computer chip pioneer Nvidia, manufacture many of their products in Asia — despite efforts by the Biden and Trump administrations to reduce U.S. dependence on essential chips made overseas.

And some forecasters say that the U.S. government’s economic data is a poor measure of the impact of AI and that alternative calculations show the current boom is an even bigger boost to economic growth.

“This is a big deal, but not the be-all and end-all,” said Joseph Politano, an economic analyst who writes the Apricitas Economics newsletter. He calculates that AI-related spending contributed about 0.2 percentage points to the 2.2 percent U.S. economic growth last year.

The AI buildup is putting real money into the pockets of some Americans and U.S. businesses. Stock market gains from AI enthusiasm are plumping up Americans’ investment portfolios.

“The two engines of today’s economy are the AI ecosystem and wealthy consumers,” Richmond Fed President Tom Barkin said in a January speech.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.

Prominent economists, including from Morgan Stanley and JPMorgan Chase, calculate that the AI buildup was directly responsible not for 92 percent or 39 percent of gains to the U.S. economy in 2025, but as little as zero.