New arXiv policy: 1-year ban for hallucinated references

Original Article ↗ Hacker News Discussion ↗

Scope of the New arXiv Policy

  • Policy (as reported in the thread):
    • One-year ban for submissions with AI-hallucinated or obviously fake references.
    • Afterward, future submissions must first be accepted at a “reputable peer‑reviewed venue.”
    • Authors are held fully responsible for all content, regardless of tools used.
  • Some note this is not yet clearly documented on arXiv’s public policies; may be planned or evolving.

Arguments Strongly Supporting the Policy

  • Fake or non-existent references are framed as:
    • Equivalent to fraud or at least gross negligence, not a minor error.
    • A basic failure of scholarly standards: verifying that every cited work exists and is relevant is “table stakes.”
    • A signal that the rest of the paper (data, analysis, conclusions) may also be unreliable.
  • References are seen as core to scientific work, not cosmetic; sloppiness wastes readers’ and reviewers’ time.
  • A ban is viewed as:
    • A necessary deterrent in an era where LLMs make slop easy to produce.
    • A way to protect the scientific record and increase arXiv’s value.
  • Many argue: if AI is used correctly and outputs are checked, this policy imposes no cost on honest researchers.

Criticism and Concerns

  • Some see the penalty as excessive, especially the ongoing requirement for prior peer‑reviewed acceptance, which is interpreted by some as a de facto lifetime constraint.
  • Concern that:
    • arXiv is meant for preprints and early dissemination; tying it to traditional peer review undercuts its purpose.
    • Peer review is much harder to clear than arXiv’s bar, potentially locking out less-established researchers.
  • Debate over whether one hallucinated citation proves “fraud” vs. mere carelessness, especially in multi‑author or last‑minute-edit scenarios.

Enforcement and Practical Issues

  • arXiv cannot comprehensively check references; enforcement likely relies on:
    • Automated tools (DOI checks, citation matchers).
    • Reader reports and spot checks.
  • Some propose LLMs and specialized tools for citation verification, though others insist database/HTTP checks are needed due to high stakes.

Broader Themes

  • Strong pushback against “AI slop” in academia; many want higher standards, not AI bans.
  • Divides appear between those enthusiastic about automated research and those worried about erosion of rigor and reviewer burden.