Write the eval before the prompt
Every LLM feature I have shipped went through the same loop: tweak the prompt, try five examples, feel good, ship, get a bug report that the five examples never covered. The fix is the one we already know from testing. Build the eval set first, from real failures, and let the prompt be the thing that has to pass it.
9 min readRead more →