Postgres LISTEN/NOTIFY does not scale

Original Article ↗ Hacker News Discussion ↗

Core issue: LISTEN/NOTIFY and global commit lock

  • Commenters reiterate the article’s finding: issuing NOTIFY inside a transaction causes a global lock at commit, effectively serializing commits that use it.
  • Several people note this behavior is not mentioned in the Postgres docs, which they find surprising for a lock-heavy feature.
  • Some ask for clarification whether only transactions that called NOTIFY are serialized; the exact scope is left somewhat unclear in the thread.

“Doesn’t scale” vs “used incorrectly”

  • Many argue the feature scales fine for typical loads and that the problem is using NOTIFY for every transaction at very high write concurrency.
  • Viewpoint 1: this is primarily a misuse and lack of due diligence (no debouncing, no grouping, no outbox), not an indictment of LISTEN/NOTIFY in general.
  • Viewpoint 2: even if misused, it’s a realistic trap; the doc and API design make it easy to step on a landmine under growth.

Design limitations of LISTEN/NOTIFY

  • No authorization model: you can’t restrict which roles may LISTEN/NOTIFY on which channels.
  • Payload constraints and typing are absent; some wish for a CREATE CHANNEL-style DDL with authz and type checking.
  • LISTEN/NOTIFY is lossy (missed messages if nobody is listening) and awkward with connection poolers.
  • A benchmark is cited: latency grows ~linearly with number of idle listeners (O(N) backend scan), making many listeners problematic.

Alternative patterns within Postgres

  • Notify outside the main transaction, with:
    • A transactional outbox table plus a sweeper, accepting duplicates and eventual consistency.
    • Notifications used only as “wake up” hints; consumers then poll canonical tables.
  • Use tables as queues via SELECT … FOR UPDATE SKIP LOCKED, backoff polling, and sometimes partitioning to reduce dead-tuple impact.
  • Multiple people describe hard problems here: dead tuples, autovacuum tuning, planner misestimation, HOT updates, and queue-table design. Extensions like pgmq/pgflow and higher-level frameworks (e.g., DBOS, Chancy) are mentioned.

WAL / logical decoding and external systems

  • Several recommend reading the WAL / logical replication instead of LISTEN/NOTIFY, using:
    • pg_logical_emit_message() for an outbox without table bloat.
    • Logical decoding consumers (e.g., via Debezium) or streaming replication.
  • Others advocate dedicated infra for pub/sub and queues: Redis, Kafka, NATS, RabbitMQ, MQTT, SQS, Centrifugo, etc., often with Postgres as system-of-record only.

Business logic in DB vs app

  • Long sub-thread debates:
    • Pro-DB: constraints, triggers, and stored procedures centralize invariants and protect against misbehaving apps.
    • Pro-app: triggers and in-DB logic can be opaque, surprising, and hard to scale; RDBMS should not be overused as message queues or workflow engines.
  • Consensus: use DB features for data integrity and some lifecycle logic, but be cautious putting high-volume messaging or broad business workflows entirely inside Postgres.

Broader scaling lessons

  • Many say LISTEN/NOTIFY is fine for low-to-moderate scale and very convenient when you “just want one dependency.”
  • Repeated theme: start simple (often Postgres), but know where it breaks—queueing, pub/sub, and very write-heavy workloads are common cliffs that eventually call for specialized components.