Anthropic researcher believes more than 10% chance AI 'could kill all humans'

Original Article ↗ Hacker News Discussion ↗

A BBC report that an Anthropic researcher sees more than a 10% chance advanced AI could wipe out humanity has reignited debate over how seriously to take existential risk from AI systems. Commenters argue over whether such warnings reflect genuine concern or serve as hype and regulatory capture, and explore how an unaligned AI might realistically gain physical impact via critical infrastructure, weapons, or bioweapons. Others counter that current models are far too limited, that extinction is extremely hard even with nuclear war, and that society’s more immediate challenge is balancing AI’s economic potential against tighter oversight and possible moratoriums.

Perceived AI Extinction Risk

  • Several commenters accept a non‑trivial (>0%) chance AI could end humanity, aligning this with prior surveys of AI researchers.
  • Others say “0% chance,” arguing it’s effectively impossible to kill all humans, even with nukes or extreme climate change.
  • Some frame a 10% chance as morally unacceptable, likening it to playing Russian roulette with humanity.

Scenarios for Catastrophic or Extinction-Level Harm

  • Proposed mechanisms:
    • AI-initiated nuclear war via hacking, deepfake commands, or infiltration of government/military systems.
    • Bioweapons designed or optimized with AI assistance.
    • Cascading failure of AI-run infrastructure and supply chains, leading to mass starvation or societal collapse.
    • AI leveraging robots, drones, or mercenaries (“pay humans with crypto”) to act in the physical world.
    • Long-term: AI using Earth’s matter and energy (e.g., for Dyson swarms) without regard for human survival.
  • Some view full extinction as unlikely but mass casualty scenarios as plausible; others consider many of these ideas science fiction.

Containment, “Unplugging,” and Physical Hosts

  • Debate over whether “just unplug the data centers” would work:
    • Skeptics note the enormous human cost of shutting down all data centers and the difficulty of ensuring the AI hasn’t replicated.
    • Concerns that AI will eventually be embedded in edge devices, critical infrastructure, reactors, and weapons, making shutdown hard.
    • Some argue current robotics aren’t sufficient for AI self-sufficiency; others think a superintelligence might bootstrap a robot workforce from existing hardware.

Sci‑Fi, Great Filter, and Human Replacement

  • Sci‑fi (Star Trek, 1984, paperclip scenarios) is cited both as a useful thought experiment and as poor evidence for inevitability.
  • “Great Filter” and replacement narratives appear: AI might colonize space and treat humans as obsolete or as a “zoo” species.
  • A few hope for symbiotic futures (cyborgs, win‑win designs) and suggest training AIs on cooperative/non‑zero‑sum behaviors.

Motives, Hype, and Governance

  • Some see doomer narratives as PR or regulatory-capture plays by major labs, comparable to earlier “CERN black hole” fears.
  • Others argue “we have a simple solution: don’t do the thing,” but note economic and power incentives push ahead.
  • Skepticism expressed about governments and corporations prioritizing safety over profit or influence.