Building durable workflows on Postgres
Postgres as Durable Workflow Engine
- Many commenters praise how far you can go with “just Postgres,” using it for durable queues, workflows, notifications, even vector search and time‑series.
- Several teams report success building durable execution and complex notification logic on top of Postgres, highlighting low cost and simplicity when it’s already a core dependency.
- Some use it specifically to unify data and workflow state in one place, deferring specialized tooling until scale truly demands it.
Locking, Queues, and Scaling Concerns
- Common patterns include
SELECT … FOR UPDATE SKIP LOCKEDand advisory locks for distributed workers. - There is disagreement on which pattern scales better; newer advice may favor
SELECT … SKIP LOCKED, but people call for real benchmarks. - Others warn that SKIP LOCKED can cause vacuum issues and table bloat as worker counts grow.
- LISTEN/NOTIFY is seen as fragile and not scalable on busy instances, though upcoming Postgres versions are expected to improve it.
DBOS, Temporal, and Other Workflow Systems
- Users compare DBOS, Temporal, Restate, Cloudflare Workflows, and others.
- DBOS is described as less complex than Temporal but limited by Postgres throughput; its strength is atomic messaging in the same DB transaction as business logic.
- Temporal draws sharply mixed reviews: SDK and concepts are praised; operational footprint, scaling with Cassandra, and sales tactics are heavily criticized by some, while others question those cost claims.
- Some prefer to copy Temporal’s ideas and reimplement a simpler variant directly in Postgres.
Build vs Buy and the Complexity of “Rolling Your Own”
- Multiple comments note that homegrown “simple queues” often accrete retries, backoff, timeouts, cancellation, idempotency, visibility, and tooling, becoming a partial workflow engine.
- Advocates of specialized systems argue this reliability layer is non‑trivial; skeptics say queues plus idempotent jobs and a status column are enough for most use cases.
Postgres‑Everywhere vs Appropriate Tools
- One camp argues: start with Postgres for almost everything and only add systems when truly needed.
- Others call this “religious,” pointing out Postgres cost and operational pain at TB‑scale or very high throughput, and recommending purpose‑built stores (ClickHouse, etc.) for heavy analytics or logs.
- There’s broad agreement that over‑engineering for hypothetical “web scale” is as harmful as under‑engineering.