Building a highly-available web service without a database
A startup blog post describing a “database-free” architecture—keeping all state in RAM and replicating it across nodes with Raft—has prompted extensive scrutiny of whether this really simplifies anything compared to using existing databases. Commenters argue that the system is effectively a custom database with its own transaction log, durability, and replication challenges, and warn about long‑term maintenance, migration, and hiring costs. Others see value in experimenting with in‑memory and Raft-based designs for developer productivity in early-stage products, but generally recommend proven tools like Postgres, SQLite, or Redis for most real-world workloads.
Reinventing the Database
- Many commenters argue the system is effectively “a database”: in‑memory objects plus snapshots, transaction log, indices, and Raft replication.
- Several point out that if you’re writing a snapshotter, WAL/transaction log, recovery, indexing, and migrations, you’ve recreated core DB functionality, often with weaker durability and fewer tools.
- Others note this pattern is well known (Prevayler, bknr.datastore, NUMA systems, AS/400, actor/event-sourcing systems, Erlang/Orleans, HashiCorp tools).
Why Not Use Existing Databases?
- Strong push toward “boring tech”: Postgres/MySQL/SQLite/Redis are viewed as simpler, better understood, easier to hire for, and battle-tested.
- Multiple people say colocation (DB on same machine or Unix socket, or SQLite + replication tools like Litestream/rqlite/dqlite) would deliver similar latency without custom infra.
- Some say Redis is ideal as a cache, not a primary durable store; others report long, successful Redis or SQLite use in production.
Complexity, Reliability, and Operations
- Critics highlight hidden complexity: Raft round-trips, concurrency via mutexes, crash consistency, log corruption, snapshots, upgrades, and cross-version compatibility.
- Concerns about incident response, recovery from edge cases, and implementing safe migrations without SQL tooling.
- Some infrastructure engineers affirm the architecture is viable but emphasize upgrade and evolution hazards.
Startup Phases and Developer Productivity
- The author’s framing: optimize for early “Explore” phase with in‑memory store; later “Expand” by adding Raft rather than rewriting to a transactional DB.
- Several disagree, saying relational DBs actually make exploration easier due to schemas, migrations, and powerful queries; switching later is common and manageable.
Performance, Cost, and RAM
- Debate over whether saving serialization/DB round-trips meaningfully improves performance for such workloads.
- Some worry about RAM cost and inefficiency if replicated in-memory state grows large; others stress that modern machines can hold substantial RAM and many apps never outgrow a single box.
Attitudes Toward Experimentation
- Many call this textbook overengineering and a maintenance nightmare for typical web apps.
- A minority welcome experimentation and non-SQL approaches, seeing value in challenging defaults, especially when built on existing Raft and datastore libraries rather than fully from scratch.