"We ran out of columns"

Original Article ↗ Hacker News Discussion ↗

Programmers swap war stories about massive legacy systems built on improvised database schemas, hand-rolled ID generators, and decades-old tooling that somehow still power real businesses. Many contrast the freedom and creativity of “ugly but effective” codebases with modern best practices, arguing that constraints, time pressure, and business needs often justify kludges—even as they create long-term technical debt and painful migrations. A recurring theme is the trade-off between careful up‑front design (schemas, normalization, migrations) and flexible, fast iteration using blobs, JSON, or ad‑hoc structures.

Playful coding vs “enterprise brain”

  • Many relate to missing the freedom of early, expectation‑free coding and now feeling unable to “do something poorly on my terms.”
  • Suggested coping strategies:
    • Maintain a “junk drawer” repo where quality rules are explicitly relaxed.
    • Develop a conscious “kludge mode” for prototypes, Emacs hacks, or personal OSS where shortcuts are allowed.
    • Define very small, fixed scopes so you can ignore scalability/maintenance concerns.
    • Accept that some of your most useful work will be ugly but effective.

Schemas, JSON blobs, and migrations

  • One camp still prefers careful relational schemas, constraints, and normalization (3NF) for safety, performance, and clarity.
  • Another camp increasingly uses “ID + JSON blob” (often in SQLite/Postgres):
    • Pros: very flexible, easy to evolve fields, avoids frequent migrations, good for semi‑structured or per‑user data, easy “search everywhere.”
    • Cons: weak type checking, harder queries, painful bulk updates, and long‑term complexity when you must support many data versions.
  • Several hybrid strategies:
    • Store raw vendor or “odd‑shaped” data as JSON, but extract frequently used or indexed fields into normal columns.
    • Use Protobuf or validation layers atop JSON and periodic checks or DB extensions to enforce schema at commit time.
  • Some argue fear of migrations is overstated: with well‑modeled entities, large disruptive migrations are rare; others counter that in messy/uncertain domains schema overhauls are common.

Relational DB features and auth

  • Multiple comments highlight underused DB features:
    • Row‑level security and rich access control (especially Postgres).
    • Computed columns over JSON for indexing and constraints.
  • There’s debate over pushing type and access logic into the DB vs the app, with trade‑offs in safety vs flexibility.

Sequences, global IDs, and column limits

  • Distinction between:
    • Built‑in sequences/identity columns (safe, fast, handle concurrency).
    • Ad‑hoc “sequence tables” updated manually, which are fragile and slow.
  • Some systems used a single global sequence across many tables so related rows shared the same ID; seen as unusual but occasionally useful.
  • Hitting column limits led some systems to split one logical table across multiple tables, or to adopt EAV/JSON approaches; many find wide, denormalized tables a smell, but note law or reporting requirements sometimes push toward them.

Legacy horror stories and lessons

  • Numerous anecdotes: gigantic Perl scripts, COBOL+Java with JAR patching, Access/VBA monstrosities, VB6/ASP, file‑locking VCSs, no tests, live‑editing production, etc.
  • Themes:
    • These systems can be weirdly resilient and profitable.
    • They’re fertile ground for learning and impactful refactors—but also dangerous for juniors and often culturally resistant to improvement.
    • Rewrites can be more damaging than incremental cleanup, especially when driven by status/“modern tech” rather than real needs.