Losing faith in testing

Original Article ↗ Hacker News Discussion ↗

Role of Tests vs Design

  • Many argue tests can’t rescue fundamentally bad architecture; simplification and comprehensibility matter more.
  • Others report success using integration tests to first “wrap” a legacy mess, then safely refactor it, seeing tests as an enabler of simplification rather than a substitute.
  • Tests are described as a tool for confidence and change, not a guarantee of correctness.

Where Tests Provide High Value

  • Core, stable, load‑bearing components (VMs, DBs, core business logic) benefit greatly from extensive testing and fuzzing.
  • Integration and end‑to‑end tests are praised for catching real, cross‑system bugs and enabling fearless refactors, especially in multi‑service and infra‑heavy systems.
  • Tests are valuable for upgrading libraries/frameworks, preventing regressions, and preserving behavior when original authors leave.
  • Property testing and strong suites are framed as “battle‑hardening” for mature systems.

Where Tests Are Seen as Low ROI

  • Fast‑changing product features and UI polish in early‑stage startups can be over‑tested, killing velocity and leading to brittle, constantly‑failing suites.
  • Interactive tools (text editors, terminals) may get away with fewer automated tests if main paths are exercised continuously by use.
  • Some teams report shipping acceptable products with little automation, relying on manual testing, metrics, canary deploys, and rollbacks.

Cultural and Process Issues

  • Testing is often treated as religion: coverage targets, TDD dogma, and mandatory tests-for-every-line produce low‑value, mock‑heavy suites.
  • Goodhart’s law: coverage as a KPI leads to trivial or redundant tests that don’t improve quality but slow change.
  • Incentives (ticket throughput, lack of time) push devs toward “hacked‑together crap”; blaming individuals instead of systems is criticized.

Alternatives, Complements, and Strategies

  • Strong type systems eliminate many classes of bugs and reduce needed tests, but don’t replace them; types and tests are seen as complementary.
  • Manual exploratory testing is still considered crucial for UX issues, performance surprises, and unanticipated interactions.
  • Several commenters advocate treating tests as investments: focus on high‑value scenarios, core invariants, and regression protection; keep suites fast, low‑flake, and be willing to delete or avoid low‑ROI tests.