Ask HN: Why do message queue-based architectures seem less popular now?

Hacker News Discussion ↗

Perceived Decline vs Reality

  • Many commenters argue queues are still used “everywhere” (SQS, Kafka, RabbitMQ, Pub/Sub, Redis, etc.), but have become “boring infrastructure,” so fewer blog posts and talks.
  • Cloud-hosted and managed offerings (SQS, Kinesis, EventBridge, Kafka-as-a-service, SNS, etc.) hide much of the complexity, so they appear less prominently in architecture diagrams.
  • Hype cycles moved on (microservices → queues → “event-driven” → AI), but queues have largely reached the “mature, boring, productive” phase.

Impact of Microservices and Architecture Trends

  • Earlier microservices enthusiasm drove heavy queue usage; later backlash showed many teams built “distributed monoliths” and unnecessary complexity.
  • Several argue most companies are not at Google/Uber scale and can run comfortably on a monolith plus a strong database.
  • Others maintain microservices still make sense for large organizations and organizational reasons (team boundaries, ownership), with queues as glue.

Why Queues Are Still Used

  • Decoupling and resilience: protect core apps from slow/unreliable third parties, absorb traffic spikes, buffer during outages/maintenance, support retries and dead-lettering.
  • Asynchronous/background work: emails, billing, ETL/ML pipelines, analytics, IoT messaging, hardware/sensor data, workflow engines, embedded/RTOS, and enterprise integration buses.
  • Operational benefits: backpressure, observability of queue depth, backfilling, smoothing load, and simpler scaling of stateless workers.

Reasons to Avoid or Minimize Queues

  • Added complexity: distributed debugging, observability, schema evolution, message loss/duplication, ordering, and consistency with the source of truth.
  • Many “event-driven” designs underestimate sync/consistency issues and the difficulty of guaranteeing delivery; missed or out-of-order events cause subtle bugs.
  • Some see widespread queue/microservice adoption as resume-driven overengineering, especially for small teams and low-volume apps.

Alternatives and Evolutions

  • Databases + Redis: job tables, transactional outbox patterns, NOTIFY/LISTEN, Redis lists/streams often suffice at modest scale with better tooling and simplicity.
  • HTTP + service discovery: lightweight services using HTTP/JSON and async runtimes replace many queue-based RPC patterns.
  • Logs and workflow engines: log-based systems (Kafka, Pulsar) used as stateful event logs; durable execution/workflow platforms (Temporal, Step Functions, etc.) handle orchestration atop queues/streams.