Enforce idempotency with a partial unique index
A retry that inserts the same message twice is the caller's fault and your problem. Store an idempotency key, and make it unique only while the row is active, so millions of finished rows do not bloat the index.
CREATE UNIQUE INDEX queue_idem ON queue (idempotency_key)
WHERE status IN ('pending', 'processing');The second insert hits the index. Catch the conflict, return the existing id, and the retry becomes a no-op without a trip to Redis.
postgresqldistributed-systems
Longer version: the post this came from.