Skip to content

.tip

· one of the small things

Give every LLM feature a token budget

A customer once pointed a bulk export at a 400,000 row table, and the feature burned a month's budget in an afternoon. A counter in Redis, checked before every call, turns that into an alert at the hourly limit and a tripped switch at the daily one.

if (Number(await redis.get(dayKey())) >= budget.daily) disableAndPage();

Base the numbers on real usage plus headroom. They're there for the abnormal afternoon, and a normal one should never touch them.

llmreliability

Longer version: the post this came from.