Losing faith in testing
Debate over software testing is shifting from “more is better” to “only where it pays off.” Commenters compare experiences from interactive tools, startups, and large enterprise systems to argue that tests are an economic investment, not a moral obligation: indispensable for core, stable and safety‑critical logic, but often wasteful for fast‑changing UI features or prototypes. Many emphasize that code design, type systems, integration and end‑to‑end checks, and a culture of caring about quality matter at least as much as raw test coverage, and that overbuilt or brittle test suites can slow teams down and even mask deeper problems.
Role of Tests vs Design
- Many argue tests can’t rescue fundamentally bad architecture; simplification and comprehensibility matter more.
- Others report success using integration tests to first “wrap” a legacy mess, then safely refactor it, seeing tests as an enabler of simplification rather than a substitute.
- Tests are described as a tool for confidence and change, not a guarantee of correctness.
Where Tests Provide High Value
- Core, stable, load‑bearing components (VMs, DBs, core business logic) benefit greatly from extensive testing and fuzzing.
- Integration and end‑to‑end tests are praised for catching real, cross‑system bugs and enabling fearless refactors, especially in multi‑service and infra‑heavy systems.
- Tests are valuable for upgrading libraries/frameworks, preventing regressions, and preserving behavior when original authors leave.
- Property testing and strong suites are framed as “battle‑hardening” for mature systems.
Where Tests Are Seen as Low ROI
- Fast‑changing product features and UI polish in early‑stage startups can be over‑tested, killing velocity and leading to brittle, constantly‑failing suites.
- Interactive tools (text editors, terminals) may get away with fewer automated tests if main paths are exercised continuously by use.
- Some teams report shipping acceptable products with little automation, relying on manual testing, metrics, canary deploys, and rollbacks.
Cultural and Process Issues
- Testing is often treated as religion: coverage targets, TDD dogma, and mandatory tests-for-every-line produce low‑value, mock‑heavy suites.
- Goodhart’s law: coverage as a KPI leads to trivial or redundant tests that don’t improve quality but slow change.
- Incentives (ticket throughput, lack of time) push devs toward “hacked‑together crap”; blaming individuals instead of systems is criticized.
Alternatives, Complements, and Strategies
- Strong type systems eliminate many classes of bugs and reduce needed tests, but don’t replace them; types and tests are seen as complementary.
- Manual exploratory testing is still considered crucial for UX issues, performance surprises, and unanticipated interactions.
- Several commenters advocate treating tests as investments: focus on high‑value scenarios, core invariants, and regression protection; keep suites fast, low‑flake, and be willing to delete or avoid low‑ROI tests.