Dev-owned testing: Why it fails in practice and succeeds in theory
Nature of the paper and venue
- Some see it as essentially a blog post that happened to land in ACM SIGSOFT Software Engineering Notes; others note that this venue explicitly welcomes experience- and position-papers from practitioners.
- Commenters highlight it’s a thought piece with little empirical data, some citation issues, and heavy reliance on one Google blog post about flaky tests.
Dev-owned testing: success stories
- Several describe highly successful dev-owned testing setups:
- Teams set their own timelines, owned on-call, and treated tests, logs, and metrics as first-class.
- Writing tests was mandatory for every feature (including sad paths), enforced in code review.
- CI/CD was reliable and fast, with comprehensive automated tests giving confidence to deploy multiple times daily.
- In these contexts, “slow is smooth, smooth is fast” testing culture led to fewer production bugs and less need for manual verification.
Failure modes of dev-owned testing
- Common problems:
- No plan: QA removed overnight, no guidance, devs forced to test their own work under velocity pressure.
- Perverse incentives and stack ranking drive shipping features over writing tests.
- Tests written just to satisfy “definition of done,” often shallow or brittle.
- Flaky tests tolerated instead of treated as real bugs.
- Developers report giving up on tests when management doesn’t protect test suites from churn, rewrites, or deletion.
Role, quality, and status of QA
- Strong theme: QA and dev skillsets/mindsets overlap but are distinct; good QA:
- Thinks like a “breaker,” acts as independent interpreter of requirements, and advocates for users.
- Finds edge cases, usability problems, and compliance issues; often accumulates deep system knowledge.
- Experiences with QA are bimodal: some describe QA as “gold,” others as unskilled, noisy, or pure friction.
- Many argue organizations underpay and undervalue QA, leading strong testers to move into better-compensated dev/security roles, which in turn degrades average QA quality.
Organizational incentives, process, and culture
- Repeated points:
- Engineering must ultimately own quality; QA provides signal, not absolution.
- Top-down “shrink QA” decisions made for cost-cutting, without changing timelines or expectations, largely doom dev-owned testing.
- Fast release tempo, weak environments, and management pressure to cut estimates often crowd out thoughtful testing.
Scope, tooling, and environments
- Several distinguish:
- Dev tests: unit/integration, white-box, API-level checks.
- QA tests: exploratory, black-box, cross-platform/regulatory/compliance, weird edge cases.
- Good test infrastructure and production-like environments are seen as crucial; poor ones make any testing model ineffective.
Critiques of the article’s stance
- Some say the title (“fails in practice”) overreaches given the author’s narrow, Amazon-centric anecdotal base and lack of reference to broader research (e.g., DORA).
- Others agree with many observations but frame the real issue as poor planning, incentives, and culture rather than dev-owned testing itself.