Enforce idempotency with a partial unique index
A retry that inserts the same message twice is the caller's fault and your problem. Store an idempotency key and make it unique only while the row is active, so millions of finished rows don't bloat the index.
CREATE UNIQUE INDEX queue_idem ON queue (idempotency_key)
WHERE status IN ('pending', 'processing');The second insert hits the index. Catch the conflict and return the existing id, and the retry turns into a no-op with no trip to Redis.
postgresqldistributed-systems
Longer version: the post this came from.