Do you even need a database?

When Flat Files or Simple Stores Are “Enough”

  • Many argue that for early-stage or low-traffic apps (waitlists, small tools, static-ish data), JSON/JSONL, CSV, or simple key-value files are sufficient.
  • OS page cache + NVMe make sequential file I/O surprisingly fast; for read-heavy, simple ID lookups, hand-rolled binary search over a sorted file can beat SQLite in benchmarks.
  • Examples: static JSON served to the client, small internal tools, batch-style or append-only workloads, single-process apps, personal projects, and game engines with tight latency budgets.

Arguments for Using a Real Database (Especially SQLite/Postgres)

  • Repeated warning: homegrown file stores tend to grow into fragile, under-tested mini-databases (indexes, joins, constraints, WAL, crash recovery) after the fact.
  • SQLite is praised as “local data solved”: ACID semantics, tested heavily, easy backups, simple deployment, often faster than naive file I/O, and can be swapped for Postgres later.
  • Relational DBs bring schema evolution, foreign keys, uniqueness constraints, flexible queries, and better support for joins and reporting.

Reliability, Concurrency, and Operational Concerns

  • A major criticism of the article’s approach: it downplays durability, crash safety, and concurrent writes.
  • File-based approaches must handle fsync, atomic renames, directory syncing, corruption, and power-loss scenarios; links to well-known file consistency pitfalls are cited.
  • Multiple writers, cron jobs, queues, redundancy, and HA quickly push you toward a DB or other networked store.

NoSQL, KV Stores, and Alternatives

  • Several comments note that starting on NoSQL (especially DynamoDB) often ends with re-implementing relational features in the app.
  • Embedded KV stores like LevelDB are seen as good for single-key lookups but tend to accrete secondary indexes and constraints, again drifting toward an RDBMS.
  • Some advocate Postgres with JSON/JSONB as a pragmatic middle ground; others point to DuckDB or S3-as-datastore for specific analytics or blob use cases.

Meta: YAGNI, Over-Engineering, and Cloud Complexity

  • Strong sentiment against premature complexity: Kubernetes, multi-DB stacks, serverless, and managed clusters for tiny apps.
  • Counterpoint: “YAGNI” shouldn’t justify unsafe storage; databases encode decades of hard-won reliability lessons that flat files often rediscover the hard way.