Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

Original Article ↗ Hacker News Discussion ↗

Arguments over what should count as “artificial intelligence” dominate this exchange, with many contrasting today’s large language models—powerful but error-prone statistical systems—with the long-imagined goal of human‑like or “AGI” intelligence. Commenters debate whether current models genuinely reason or merely simulate it, how shifting definitions of “intelligence” and “AI” distort public expectations, and why overhyping or mythologizing the technology could lead to mismanagement, misplaced fear, and poor policy. Alongside this, there are calls for clearer terminology, digital provenance for AI outputs, and a culture of caution about both economic impacts and future existential risks.

Scope and Timeliness of the Article

  • Several note the article is ~3 years old; some think it’s still valid since modern LLMs are scaled-up versions of the same basic approach.
  • Others see posting it now as bait, given how quickly models have advanced and how defensive some users get when LLMs are criticized.

What Counts as “AI” and “Intelligence”

  • Strong disagreement over calling LLMs “AI”:
    • One side says they are advanced text generators, not autonomous intelligences; “AI” should be reserved for human-like or “AGI”-level systems.
    • Others argue the field has used “AI” for decades for everything from game AIs to NPCs, and LLMs clearly meet “artificial” + “intelligent behavior” in practice.
  • Many point out “intelligence” lacks a clear, measurable definition; people talk past each other and often move goalposts when computers master a task.
  • Some suggest abandoning “intelligence” as a scientific term and instead talking about specific capabilities or skills.

Capabilities and Failures of LLMs

  • Supporters emphasize they can solve original math problems, code, and pass exams; this can’t be meaningfully compared to calculators.
  • Skeptics highlight failure modes: hallucinations, nonsensical plans (e.g., walking vs. driving), prompt injection, tool-state confusions, language-drift, and brittle reasoning.
  • Debate over whether these failures are fundamentally unlike human errors or just another flavor of fallible reasoning.
  • Some liken current multi-call/agent setups to probabilistic algorithms: run many times until a good-enough answer appears.

Turing Test, AGI, and Anthropomorphism

  • Disagreement about whether modern models “pass” robust Turing tests, especially against motivated human judges.
  • Some see LLMs as an “alien” intelligence; others say calling this “reasoning” is just relabeling pattern-matching.
  • Concern that mythologizing and anthropomorphizing systems (marketing terms like “reasoning,” “agents”) misleads the public and policymakers.

Risk, Autonomy, and Usefulness

  • One camp: current systems are powerful tools but nowhere near autonomous takeover; “existential risk” talk feels premature or quasi-religious.
  • Another: it’s not too early to be cautious; increasing autonomy plus infrastructure access could be dangerous, especially as local, fast models improve.

Provenance, Privacy, and Governance

  • Strong interest in “digital provenance” for AI outputs—an equivalent of “View Source” that explains which training data patterns shaped behavior, to support accountability and debugging.
  • Some prefer defining privacy as control over what data is extracted, rather than vague “right to be left alone” or “not be manipulated.”
  • Several argue that naming and conceptual framing matter a lot: misnaming technologies can lead to bad expectations, hype, and policy mistakes.