#llm

A million token context window did not make the long-running agent problem go away. It moved it. The model still forgets the thing you told it an hour ago, only now it forgets it after paying for it forty times. The techniques that work, compaction, retrieval, memory files, all turn out to be the same techniques we use to manage a cache.

A model provider had a bad Tuesday and our summarisation feature spent four hours returning empty strings to customers, because nothing between the API and the user knew what to do with a 529. The feature now has a kill switch, a fallback, a budget, and a canary. Those four things are the difference between an LLM feature and an LLM demo.

I moved commit messages, log triage, PR summaries, test naming and half a dozen other small jobs from a hosted API to an open-weight model running on my laptop. It was not about cost. It was about a category of task where a 30B model on a MacBook is indistinguishable from the frontier, and where I would rather the data never left the machine.

Every team that adopted MCP hit the same wall by spring: forty tools in the context, a model that picks the wrong one, and a bill that grew with every server you added. Skills, the folder-of-markdown pattern that spread through coding agents this year, fix a different problem than MCP does. Knowing which is which decides whether your agent gets better or just bigger.

A single GitHub issue was enough to make a coding agent leak a private repo through the official MCP server. The Git MCP server fell to path traversal from a prompt alone in January. None of these were model failures. They were the same mistake we made with SQL in 2004: letting data and instructions travel on the same wire.

Every LLM feature I have shipped went through the same loop: tweak the prompt, try five examples, feel good, ship, get a bug report that the five examples never covered. The fix is the one we already know from testing. Build the eval set first, from real failures, and let the prompt be the thing that has to pass it.

The internal docs assistant answered confidently and was wrong a third of the time. Everyone blamed the model. The model was fine. The retrieval was returning the right document eleventh out of ten. Fixing it was six weeks of ordinary search engineering, and none of it involved a prompt.