AI Detectors Get It Wrong. Writers Are Being Fired Anyway

Original Article ↗ Hacker News Discussion ↗

AI-written text detectors are being used to fire writers and punish students despite high false-positive rates and no reliable way to distinguish human from machine-generated prose. Commenters argue that, because large language models are trained to mimic human writing styles and can be trivially prompted or post-edited to evade detection, current tools are closer to “snake oil” than to evidence. Many see greater harm coming from institutions and employers outsourcing judgment to opaque algorithms—creating perverse incentives for surveillance and self-defense—than from the underlying use of AI in everyday writing itself.

AI Detectors’ Reliability and Limits

  • Many argue text detectors are fundamentally flawed “snake oil”: LLMs are trained on human writing and can mimic any style, so statistical tests can’t reliably separate human vs AI.
  • OpenAI’s own abandonment of a detector is cited as evidence of unsolved accuracy issues.
  • Simple tweaks (wording, spacing, paraphrasing, prompting for specific styles) can bypass detectors.
  • Detectors tend to flag polished, grammatical, or “average” prose; one product openly warns of false positives for non‑native speakers, technical writing, and neurodivergent authors.

Real-World Harm: Students and Writers

  • Commenters describe students accused of cheating based solely on detectors or even ChatGPT’s claim “I wrote this” when asked about a passage.
  • Freelancers are reportedly suspended or fired for “excessive AI use” despite providing drafts and timestamps showing human work.
  • Some call for lawsuits or GDPR‑style rights to challenge purely algorithmic decisions; others note the high personal cost of legal fights.

Provenance, Surveillance, and Evasion

  • Proposals include keystroke‑logging editors, timestamping, and even blockchains to prove human authorship.
  • Critics see this as dehumanizing self‑surveillance that employers will eventually mandate, and note such systems are trivially faked (LLM + retyping, scripted input, hardware “finger bots”).
  • Skepticism that complex proofs of authorship will convince institutions that already over‑trust simple detectors.

Views on Using AI for Work

  • Several say the real metric should be quality and truthfulness, not whether AI was used; firing someone for using a tool seems misguided.
  • Others respond that if AI can do the job cheaply, employers will drop human writers regardless.
  • In software, many note origin of code matters less so long as it’s correct and maintainable.

AI Slop, Spam, and Information Quality

  • Widespread concern about “AI slop”: bland, SEO‑style, low‑value text flooding Q&A sites, forums, and news.
  • Some foresee an escalating arms race between AI‑generated content and AI detectors, likely degrading the internet and journalism further.

Human vs Machine Writing and Detection

  • Debate over whether people can “always tell” AI text: some insist current AI has a recognizable tone; others think good AI‑assisted writing is already indistinguishable.
  • Several predict that, as with other ML tasks, pressure from detectors will push generators to become undetectable, making origin essentially unknowable.