Skip to content

#reliability

1 post on this topic, newest first. all tags →

Every LLM feature needs a kill switch

A model provider had a bad Tuesday, and our summarisation feature spent four hours returning empty strings to customers because nothing between the API and the user knew what to do with a 529. The feature now has a kill switch, a fallback, a budget and a canary. Without those four it was a demo.

llmreliabilityfeature-flags+1 more
10 min read13 min listen