Skip to content

#sre

3 posts on this topic, newest first. all tags →

Upgrading PostgreSQL 16 to 18 with zero downtime: the whole runbook

A major version upgrade with a write pause you measure in seconds, instead of an outage you measure in hours. The plan, the preflight queries that catch what will bite you, building the new cluster from a standby with pg_createsubscriber, the sequence trap, schema changes during the window, the cutover script through PgBouncer, a rollback you can actually take, and what PostgreSQL 19 removes from this list.

postgresqldatabaseoperations+2 more
16 min read21 min listen$2.00

Letting an agent touch production

Over five months we gave an agent access to production on-call in stages: read only, then propose, then act on a short list, then act on a longer one. Each stage had to prove a specific thing before the next. The rules that came out of it fit on a page, and they aren't the ones I'd have guessed at the start.

sreai-agentson-call+1 more
8 min read10 min listen

The observability bill is a design problem

In March the telemetry bill for one service passed its compute bill. The instinct is to sample harder and log less. We went the other way: fewer, wider events, tail-based sampling that keeps every error, and a one-week retention tier for the boring 95 percent. The bill dropped by two thirds, and incidents got easier to debug.

observabilityopentelemetrycost+1 more
10 min read13 min listen