#llm/coding + #code

LLM inference, optimized for your Mac
Continuous batching and tiered KV caching, managed directly from your menu bar.

oMLX persists KV cache across a hot in-memory tier and cold SSD tier - even when context changes mid-conversation, all past context stays cached and reusable across requests, making local LLMs practical for real coding work with tools like Claude Code.

brew tap jundot/omlx https://github.com/jundot/omlx brew install jundot/omlx/omlx # Upgrade to the latest version brew update && brew upgrade omlx # Run as a background service (auto-restarts on crash) omlx start # Optional: MCP (Model Context Protocol) support /opt/homebrew/opt/omlx/libexec/bin/pip install mcp

Supports text LLMs, vision-language models (VLM), OCR models, embeddings, and rerankers on Apple Silicon.

Block-based KV cache management inspired by vLLM, with prefix sharing and Copy-on-Write. The cache operates across two tiers:

• Hot tier (RAM): Frequently accessed blocks stay in memory for fast access.
• Cold tier (SSD): When the hot cache fills up, blocks are offloaded to SSD in safetensors format. On the next request with a matching prefix, they're restored from disk instead of recomputed from scratch - even after a server restart.

For proprietary agents that don’t want to join in, good old symlinks do the job:

Fix A Broken AGENTS.md With This Prompt

If you're starting to get nervous about the AGENTS.md file in your repo, and you want to refactor it to use progressive disclosure, try copy-pasting this prompt into your coding agent:

I want you to refactor my AGENTS.md file to follow progressive disclosure principles.

Follow these steps:

1. Find contradictions: Identify any instructions that conflict with each other. For each contradiction, ask me which version I want to keep.

2. Identify the essentials: Extract only what belongs in the root AGENTS.md:

   • One-sentence project description
   • Package manager (if not npm)
   • Non-standard build/typecheck commands
   • Anything truly relevant to every single task

3. Group the rest: Organize remaining instructions into logical categories (e.g., TypeScript conventions, testing patterns, API design, Git workflow). For each group, create a separate markdown file.

4. Create the file structure: Output:

   • A minimal root AGENTS.md with markdown links to the separate files
   • Each separate file with its relevant instructions
   • A suggested docs/ folder structure

5. Flag for deletion: Identify any instructions that are:

   • Redundant (the agent already knows this)
   • Too vague to be actionable
   • Overly obvious (like "write clean code")