Text AI watermarks will always be trivial to remove
Efforts to embed invisible watermarks in AI‑generated text are drawing scrutiny as technically weak and easy to bypass, especially with local or open‑source models that can paraphrase or strip signals. Supporters argue that even imperfect schemes can deter low‑effort cheating and help institutions like universities, publishers, and governments flag obvious AI‑written content, while critics warn about false positives, privacy risks from storing prompts and outputs, and a false sense of trust in automated detectors. The debate touches on broader concerns about regulatory pressure from the EU AI Act, the future of local “unlocked” computing, and whether society should pursue AI labeling at all versus redesigning workflows to assume AI assistance is ubiquitous.
Watermarking effectiveness and ease of removal
- Many argue text watermarks are inherently fragile: paraphrasing with another (possibly local) model, reformatting, language translation, or stripping odd Unicode likely defeats them.
- Others say “better than nothing”: even if high‑effort evasion is easy, watermarking still catches low‑effort copy‑paste users and mass‑produced slop.
- Some worry it creates a false sense of security: people may over‑trust detectors and miss sophisticated forgeries.
Local models and general‑purpose computing
- Local models enable watermark‑free generation and automated watermark removal, undermining centralized schemes.
- Several commenters fear regulators will respond by restricting open weights and locking down hardware/OS ecosystems, threatening general‑purpose computing.
- Others think local use will remain niche; most people will still rely on big providers.
Alternative detection ideas and data retention
- One proposal: providers store prompts/outputs and expose a “distance” API to check if a text matches past generations.
- Pushback: privacy, legal limits on long‑term retention of sensitive data, and practical scaling issues.
- Some suggest “proof of human creation” via hardware‑attested keypress logs, but note huge privacy concerns and easy “analog loopholes” (retyping, transcription).
Use cases and beneficiaries
- Claimed beneficiaries: universities (cheating deterrence), large companies/governments (filtering spam, bogus reports), law and policy domains, peer review.
- Skeptics question real user benefit, likening this to cookie laws that create friction without solving core problems.
False positives, fairness, and access
- Strong concern about false positives in education and legal settings; even a small error rate could unjustly punish students or professionals.
- Some argue watermarking can be tuned to extremely low false positive rates; others cite past image‑hashing failures (e.g., CSAM scanners) as cautionary.
- Unclear who will get detection tools: likely paying institutions via API; general public and teachers may not have direct access.
Technical discussion (SynthID‑style schemes)
- Multiple comments describe watermarking as biasing token sampling using a secret key, producing a statistical signal over long text rather than visible markers.
- This can, in theory, survive moderate edits, splicing, and some paraphrasing, especially for longer documents; short snippets are harder.
- Still, commentators expect a cat‑and‑mouse dynamic: adversaries iteratively mutate text (possibly via tools) until detectors clear it.
Politics, regulation, and broader concerns
- Some see watermarking as regulatory theater driven by the EU AI Act and political optics, not scientific necessity.
- Others view it as a reasonable safeguard against AI‑driven scams, disinformation, and low‑quality flood, even if imperfect.
- Broader anxieties appear about surveillance, authenticated‑only text ecosystems, and a future internet saturated with AI‑generated content.