Dev-owned testing: Why it fails in practice and succeeds in theory

Nature of the paper and venue

• Some see it as essentially a blog post that happened to land in ACM SIGSOFT Software Engineering Notes; others note that this venue explicitly welcomes experience- and position-papers from practitioners.
• Commenters highlight it’s a thought piece with little empirical data, some citation issues, and heavy reliance on one Google blog post about flaky tests.

Dev-owned testing: success stories

• Several describe highly successful dev-owned testing setups:
  • Teams set their own timelines, owned on-call, and treated tests, logs, and metrics as first-class.
  • Writing tests was mandatory for every feature (including sad paths), enforced in code review.
  • CI/CD was reliable and fast, with comprehensive automated tests giving confidence to deploy multiple times daily.
• In these contexts, “slow is smooth, smooth is fast” testing culture led to fewer production bugs and less need for manual verification.

Failure modes of dev-owned testing

• Common problems:
  • No plan: QA removed overnight, no guidance, devs forced to test their own work under velocity pressure.
  • Perverse incentives and stack ranking drive shipping features over writing tests.
  • Tests written just to satisfy “definition of done,” often shallow or brittle.
  • Flaky tests tolerated instead of treated as real bugs.
• Developers report giving up on tests when management doesn’t protect test suites from churn, rewrites, or deletion.

Role, quality, and status of QA

• Strong theme: QA and dev skillsets/mindsets overlap but are distinct; good QA:
  • Thinks like a “breaker,” acts as independent interpreter of requirements, and advocates for users.
  • Finds edge cases, usability problems, and compliance issues; often accumulates deep system knowledge.
• Experiences with QA are bimodal: some describe QA as “gold,” others as unskilled, noisy, or pure friction.
• Many argue organizations underpay and undervalue QA, leading strong testers to move into better-compensated dev/security roles, which in turn degrades average QA quality.

Organizational incentives, process, and culture

• Repeated points:
  • Engineering must ultimately own quality; QA provides signal, not absolution.
  • Top-down “shrink QA” decisions made for cost-cutting, without changing timelines or expectations, largely doom dev-owned testing.
  • Fast release tempo, weak environments, and management pressure to cut estimates often crowd out thoughtful testing.

Scope, tooling, and environments

• Several distinguish:
  • Dev tests: unit/integration, white-box, API-level checks.
  • QA tests: exploratory, black-box, cross-platform/regulatory/compliance, weird edge cases.
• Good test infrastructure and production-like environments are seen as crucial; poor ones make any testing model ineffective.

Critiques of the article’s stance

• Some say the title (“fails in practice”) overreaches given the author’s narrow, Amazon-centric anecdotal base and lack of reference to broader research (e.g., DORA).
• Others agree with many observations but frame the real issue as poor planning, incentives, and culture rather than dev-owned testing itself.