Databases and why their complexity is now unnecessary

Original Article ↗ Hacker News Discussion ↗

A blog post arguing that traditional relational databases are overly complex and fundamentally flawed has prompted debate over when their constraints are actually a strength. Commenters contrast the simplicity of “one big SQL database” for most startups and business systems with the needs of petabyte‑scale, event‑driven architectures, where logs plus materialized views and tools like Rama, Kafka, or Datomic can shine. Many remain skeptical of Rama’s claims to replace databases—citing operational complexity, JVM lock‑in, and hard‑won lessons from event sourcing—while acknowledging that better tooling for schema evolution, indexing, and large‑scale data processing is still an unsolved problem.

Single vs. multiple databases

  • Strong support for “one big database” as a superpower: simpler semantics, no distributed transactions, easier debugging, and often enough scale for most companies.
  • Counterpoints:
    • You inevitably have at least one other datastore for infrastructure (e.g., etcd/ZooKeeper/K8s metadata).
    • At larger scale or stricter isolation needs, splitting data (by service, tenant, or workload) can reduce blast radius and enable independent evolution.
    • For many startups, a single vertically scaled RDBMS + a couple replicas is sufficient; horizontal sharding is often premature.

Relational models, schemas, and complexity

  • Some argue any domain can be modeled cleanly with tuples and relations; performance, not expressiveness, is the real limit.
  • Others stress restrictive schemas and normalization are a feature: they force thinking, protect data quality, and enable powerful querying.
  • Complaints focus more on schema evolution, migration risk, and ORM leakage than on the relational model itself.

Event sourcing + materialized views

  • Many point out most serious databases are already “materialized views over a log” (WAL/binlog).
  • Event sourcing is praised in high‑scale, data‑engineering‑heavy environments, but:
    • Many report it adds boilerplate, cognitive load, versioning headaches, GDPR/anonymization issues, and painful debugging.
    • Several say they’ve regretted adopting it outside narrow, well‑justified use cases.
  • Others report success when used selectively, often with Kafka/CDC/outbox patterns and traditional RDBMS as the canonical store.

Rama: architecture and reactions

  • Rama is described as: append‑only “depots” (event logs) + distributed dataflows that build arbitrary “PStates” (materialized indexes) + query topologies.
  • It runs on the JVM with Java and Clojure APIs, aims to replace the usual “Postgres + queue + search + ETL” stack, and claims strong consistency, ACID‑like semantics, and high scalability.
  • Enthusiastic comments: like the cohesive model, built‑in telemetry, and tight integration of ingestion, processing, and querying.
  • Skepticism:
    • Heavy marketing (“databases unnecessary”, “Twitter‑scale in 100x less code”) feels overstated; demo is synthetic, not a real production migration.
    • API looks like a DSL embedded in Java; JVM‑only is a barrier; learning curve and mental model are unclear.
    • Doubts that it simplifies typical business apps (e.g., shopping carts, accounting) versus a good RDBMS.

Global mutable state, transactions, and evolution

  • Agreement that global mutable state is inherently unavoidable; event logs still mutate as new events arrive.
  • RDBMS transactions and constraints remain highly valued for money‑like workloads; some see Rama’s model as just relocating, not removing, complexity.
  • Many wish for better tools for schema evolution, blue/green migrations, and “schema as code,” regardless of whether they use databases or event logs.