#llm/coding

Public notes from activescott tagged with #llm/coding

Saturday, September 12, 2026

Friday, September 4, 2026

LLM inference, optimized for your Mac
Continuous batching and tiered KV caching, managed directly from your menu bar.

oMLX persists KV cache across a hot in-memory tier and cold SSD tier - even when context changes mid-conversation, all past context stays cached and reusable across requests, making local LLMs practical for real coding work with tools like Claude Code.

brew tap jundot/omlx https://github.com/jundot/omlx brew install jundot/omlx/omlx # Upgrade to the latest version brew update && brew upgrade omlx # Run as a background service (auto-restarts on crash) omlx start # Optional: MCP (Model Context Protocol) support /opt/homebrew/opt/omlx/libexec/bin/pip install mcp

Supports text LLMs, vision-language models (VLM), OCR models, embeddings, and rerankers on Apple Silicon.

Block-based KV cache management inspired by vLLM, with prefix sharing and Copy-on-Write. The cache operates across two tiers:

  • Hot tier (RAM): Frequently accessed blocks stay in memory for fast access.
  • Cold tier (SSD): When the hot cache fills up, blocks are offloaded to SSD in safetensors format. On the next request with a matching prefix, they're restored from disk instead of recomputed from scratch - even after a server restart.

Wednesday, February 18, 2026

Fix A Broken AGENTS.md With This Prompt

If you're starting to get nervous about the AGENTS.md file in your repo, and you want to refactor it to use progressive disclosure, try copy-pasting this prompt into your coding agent:

I want you to refactor my AGENTS.md file to follow progressive disclosure principles.

Follow these steps:

  1. Find contradictions: Identify any instructions that conflict with each other. For each contradiction, ask me which version I want to keep.

  2. Identify the essentials: Extract only what belongs in the root AGENTS.md:

    • One-sentence project description
    • Package manager (if not npm)
    • Non-standard build/typecheck commands
    • Anything truly relevant to every single task
  3. Group the rest: Organize remaining instructions into logical categories (e.g., TypeScript conventions, testing patterns, API design, Git workflow). For each group, create a separate markdown file.

  4. Create the file structure: Output:

    • A minimal root AGENTS.md with markdown links to the separate files
    • Each separate file with its relevant instructions
    • A suggested docs/ folder structure
  5. Flag for deletion: Identify any instructions that are:

    • Redundant (the agent already knows this)
    • Too vague to be actionable
    • Overly obvious (like "write clean code")

Wednesday, February 4, 2026

Tuesday, February 3, 2026

Wednesday, January 7, 2026

For every complex task, create THREE files:

task_plan.md → Track phases and progress notes.md → Store research and findings [deliverable].md → Final output

The Loop

  1. Create task_plan.md with goal and phases
  2. Research → save to notes.md → update task_plan.md
  3. Read notes.md → create deliverable → update task_plan.md
  4. Deliver final output

Key insight: By reading task_plan.md before each decision, goals stay in the attention window. This is how Manus handles ~50 tool calls without losing track.

Sunday, January 4, 2026

I'm not joking and this isn't funny. We have been trying to build distributed agent orchestrators at Google since last year. There are various options, not everyone is aligned... I gave Claude Code a description of the problem, it generated what we built last year in an hour.

Monday, December 8, 2025

We introduce the Berkeley Function Calling Leaderboard (BFCL), the first comprehensive and executable function call evaluation dedicated to assessing Large Language Models' (LLMs) ability to invoke functions. Unlike previous evaluations, BFCL accounts for various forms of function calls, diverse scenarios, and executability.