.tip

Jun 5, 2026 · one of the small things

Thirty real inputs before the first prompt

Collect thirty real inputs and decide what a correct output is for each. Write the scorer. Get a number for a trivial prompt. Then write the real prompt and treat it as code that has to pass.

evals/tickets.jsonl   one case per line: input, expected, source
evals/run.ts          prints accuracy and the failures

Every correction from production goes into the set. The prompt is disposable. The set is the asset.

ai-engineeringtesting

Longer version: the post this came from.

~/tips/today — 2026-09-04

~cat tips/today

Session cookies: HttpOnly, Secure, SameSite=Lax

securityhttp

Three attributes, each closing a class of attack. HttpOnly keeps scripts from reading it. Secure keeps it off plain HTTP. SameSite=Lax stops it being sent on cross-site POSTs, which is most CSRF.

Set-Cookie: session=...; HttpOnly; Secure; SameSite=Lax; Path=/; Max-Age=1209600

Strict breaks arriving from an external link while logged in. Lax is the default worth keeping.

↵ open y yesterday l ls /tips

esc close