Building a highly-available web service without a database

Original Article ↗ Hacker News Discussion ↗

Reinventing the Database

  • Many commenters argue the system is effectively “a database”: in‑memory objects plus snapshots, transaction log, indices, and Raft replication.
  • Several point out that if you’re writing a snapshotter, WAL/transaction log, recovery, indexing, and migrations, you’ve recreated core DB functionality, often with weaker durability and fewer tools.
  • Others note this pattern is well known (Prevayler, bknr.datastore, NUMA systems, AS/400, actor/event-sourcing systems, Erlang/Orleans, HashiCorp tools).

Why Not Use Existing Databases?

  • Strong push toward “boring tech”: Postgres/MySQL/SQLite/Redis are viewed as simpler, better understood, easier to hire for, and battle-tested.
  • Multiple people say colocation (DB on same machine or Unix socket, or SQLite + replication tools like Litestream/rqlite/dqlite) would deliver similar latency without custom infra.
  • Some say Redis is ideal as a cache, not a primary durable store; others report long, successful Redis or SQLite use in production.

Complexity, Reliability, and Operations

  • Critics highlight hidden complexity: Raft round-trips, concurrency via mutexes, crash consistency, log corruption, snapshots, upgrades, and cross-version compatibility.
  • Concerns about incident response, recovery from edge cases, and implementing safe migrations without SQL tooling.
  • Some infrastructure engineers affirm the architecture is viable but emphasize upgrade and evolution hazards.

Startup Phases and Developer Productivity

  • The author’s framing: optimize for early “Explore” phase with in‑memory store; later “Expand” by adding Raft rather than rewriting to a transactional DB.
  • Several disagree, saying relational DBs actually make exploration easier due to schemas, migrations, and powerful queries; switching later is common and manageable.

Performance, Cost, and RAM

  • Debate over whether saving serialization/DB round-trips meaningfully improves performance for such workloads.
  • Some worry about RAM cost and inefficiency if replicated in-memory state grows large; others stress that modern machines can hold substantial RAM and many apps never outgrow a single box.

Attitudes Toward Experimentation

  • Many call this textbook overengineering and a maintenance nightmare for typical web apps.
  • A minority welcome experimentation and non-SQL approaches, seeing value in challenging defaults, especially when built on existing Raft and datastore libraries rather than fully from scratch.