Skip to content

#ai-engineering

2 posts on this topic, newest first. all tags →

Context engineering is cache management

A million token context window didn't make the long-running agent problem go away, it just moved it. The model still forgets what you told it an hour ago, only now it forgets after paying for it forty times. The techniques that work, compaction, retrieval, memory files, are the same ones we use to manage a cache.

llmai-agentsarchitecture+1 more
8 min read10 min listen

Write the eval before the prompt

Every LLM feature I've shipped went through the same loop: tweak the prompt, try five examples, feel good, ship, then get a bug report the five examples never covered. The fix is one we already know from testing. Build the eval set first, from real failures, and make the prompt the thing that has to pass it.

llmtestingai-engineering+1 more
9 min read11 min listen