RabbitMQ 4.0

Original Article ↗ Hacker News Discussion ↗

RabbitMQ 4.0 introduces a native AMQP 1.0 implementation, enhanced quorum queues, and a new Khepri metadata store aimed at improving scalability and reliability for messaging and streaming workloads. Commenters compare RabbitMQ with systems like NATS, Kafka, Redis, NSQ, and cloud services such as AWS SNS/SQS, weighing trade-offs in complexity, performance, routing features, persistence models, and operational burden. A recurring theme is that RabbitMQ is powerful and mature but can be difficult to operate correctly, making its value highly dependent on scale, team expertise, and whether managed services are an acceptable alternative.

New features in RabbitMQ 4.0

  • Native AMQP 1.0 support (no longer a plugin), seen as enabling easier replacement for services like Azure Service Bus behind abstractions.
  • New quorum queue capabilities and a new metadata store (Khepri) replacing Mnesia for configuration/state (vhosts, users, queue definitions, etc.), expected to improve scalability and netsplit handling.
  • Stream queues give Kafka‑like behavior: high throughput, retention, replay, and Kafka-style topic semantics in RabbitMQ.
  • Native MQTT and WebSocket MQTT support emphasized, including for large numbers of connected devices.

Scalability and operational behavior

  • Many argue RabbitMQ “scales fine” for most real‑world use, with examples of tens of thousands of messages per second and very large user bases.
  • Others report footguns: clusters slowing down when adding nodes (due to replication of durable messages), and performance degradation when queues grow unbounded.
  • Quorum queues are cited as addressing some older clustering trade‑offs; message replication can be bounded.
  • Mnesia-related slowdowns under load are reported historically; Khepri is expected to improve metadata handling but not directly message throughput.

RabbitMQ vs NATS and other brokers

  • NATS is frequently praised as simpler, lighter, and easier to deploy, especially for ephemeral pub/sub and RPC; RabbitMQ is seen as heavier but stronger at complex routing and “postal system” style queuing.
  • JetStream (NATS) offers persistence, streams, KV, and blob store, competing with RabbitMQ streams/federation, but some argue RabbitMQ still has richer advanced routing and MQ‑to‑MQ federation.
  • NSQ and Beanstalkd are mentioned as simpler alternatives; Beanstalkd is seen as extremely minimal but with reported stability issues in some deployments.

RabbitMQ vs Redis / DB / managed queues

  • Compared with Redis-backed queues and DB tables used as queues, RabbitMQ is seen as more featureful and better for high throughput, complex routing, and multi-language ecosystems.
  • Some teams moved from Redis queues to RabbitMQ for throughput, priorities, and routing.
  • For small workloads (e.g., ~20 events/s on SNS/SQS), many see no reason to switch; debate centers on cost (managed queues vs running your own broker) versus engineering time and operational burden.

Developer experience, learning curve, and ecosystem

  • RabbitMQ is described as powerful but easy to “hold wrong,” with many docs to absorb and multiple footguns; Kubernetes deployments are possible but finicky.
  • Some find it more trouble than it’s worth; others describe it as a “secret weapon” if properly understood.
  • Confusion persists about what RabbitMQ is from its website copy; several note that terms like “messaging and streaming broker” are not self‑explanatory to newcomers.
  • Celery uses RabbitMQ by default; there were recent limitations with quorum queues that have been patched.

Support, stability, and versioning concerns

  • Concerns are raised about RabbitMQ’s future under VMware/Broadcom, including hints that “free support” may be reduced, but details are unclear.
  • One commenter reports a negative experience with semantic versioning in RabbitMQ client libraries (breaking changes without major version bumps) and contentious maintainer responses.
  • Interest is expressed in whether RabbitMQ 4.0 passes Jepsen-style consistency tests, but no answer is provided in the thread (status unclear).