Building durable workflows on Postgres

Postgres as Durable Workflow Engine

  • Many commenters praise how far you can go with “just Postgres,” using it for durable queues, workflows, notifications, even vector search and time‑series.
  • Several teams report success building durable execution and complex notification logic on top of Postgres, highlighting low cost and simplicity when it’s already a core dependency.
  • Some use it specifically to unify data and workflow state in one place, deferring specialized tooling until scale truly demands it.

Locking, Queues, and Scaling Concerns

  • Common patterns include SELECT … FOR UPDATE SKIP LOCKED and advisory locks for distributed workers.
  • There is disagreement on which pattern scales better; newer advice may favor SELECT … SKIP LOCKED, but people call for real benchmarks.
  • Others warn that SKIP LOCKED can cause vacuum issues and table bloat as worker counts grow.
  • LISTEN/NOTIFY is seen as fragile and not scalable on busy instances, though upcoming Postgres versions are expected to improve it.

DBOS, Temporal, and Other Workflow Systems

  • Users compare DBOS, Temporal, Restate, Cloudflare Workflows, and others.
  • DBOS is described as less complex than Temporal but limited by Postgres throughput; its strength is atomic messaging in the same DB transaction as business logic.
  • Temporal draws sharply mixed reviews: SDK and concepts are praised; operational footprint, scaling with Cassandra, and sales tactics are heavily criticized by some, while others question those cost claims.
  • Some prefer to copy Temporal’s ideas and reimplement a simpler variant directly in Postgres.

Build vs Buy and the Complexity of “Rolling Your Own”

  • Multiple comments note that homegrown “simple queues” often accrete retries, backoff, timeouts, cancellation, idempotency, visibility, and tooling, becoming a partial workflow engine.
  • Advocates of specialized systems argue this reliability layer is non‑trivial; skeptics say queues plus idempotent jobs and a status column are enough for most use cases.

Postgres‑Everywhere vs Appropriate Tools

  • One camp argues: start with Postgres for almost everything and only add systems when truly needed.
  • Others call this “religious,” pointing out Postgres cost and operational pain at TB‑scale or very high throughput, and recommending purpose‑built stores (ClickHouse, etc.) for heavy analytics or logs.
  • There’s broad agreement that over‑engineering for hypothetical “web scale” is as harmful as under‑engineering.