#testing

1 post on this topic, newest first. all tags →

Write the eval before the prompt

Every LLM feature I've shipped went through the same loop: tweak the prompt, try five examples, feel good, ship, then get a bug report the five examples never covered. The fix is one we already know from testing. Build the eval set first, from real failures, and make the prompt the thing that has to pass it.

llmtestingai-engineering+1 more

Apr 7, 20269 min11 min

~/tips/today — 2026-10-19

~cat tips/today

Never build an index on a live table without CONCURRENTLY

postgresqlmigrations

A plain CREATE INDEX takes a lock that blocks every write to the table for the whole build. On a big table that means minutes of outage that look like a hung app.

CREATE INDEX CONCURRENTLY orders_customer_idx ON orders (customer_id);

It's slower and can't run inside a transaction, so you'll have to tell your migration tool. If it fails halfway, it leaves an invalid index behind, so check pg_index.indisvalid and drop it before you retry.

↵ open y yesterday l ls /tips

esc close