What Comes After AGI?

Debate over DeepMind’s “What Comes After AGI?” essay quickly centers on whether artificial general intelligence is actually close, or whether current large language models are just powerful but narrow pattern-matchers being oversold by Big Tech marketing. Commenters argue over definitions of AGI and ASI, with some claiming we already have human-level capability on many desk tasks and others insisting that true generality, continual learning, embodiment, and real-world decision-making are still missing. Many worry more about near-term impacts—mass white‑collar unemployment, political backlash, and concentration of power—than speculative superintelligence, while noting that safety and alignment issues receive far less concrete treatment than scaling curves and investment projections.

Skepticism about “AGI is near” framing

  • Many reject the article’s premise that “AGI is coming within reach,” calling it marketing or “propaganda.”
  • Pointed out: if labs had true AGI, they’d use it to autonomously make money or run their own companies, not mainly sell API access.
  • Some see DeepMind/Big Tech as over-extrapolating recent progress (investment curves, scaling laws) in ways reminiscent of 1999 dot‑com hype.

Definitions: AGI, ASI, and “intelligence”

  • Strong disagreement on what AGI means; several say the term has always been vague and the goalposts keep moving.
  • Competing definitions:
    • Behavioral: “as good as most humans at most (desk) tasks.”
    • Stronger: can do any cognitive task at human level, with continual learning, self‑modified “weights,” curiosity, and autonomous goal-setting.
    • Others emphasize embodiment, long‑horizon real‑world action, or the ability to fully substitute for a human in daily life.
  • Some argue AGI and ASI blur together: once human-level is reached at scale, superintelligence is almost inevitable; others think human-level might be a practical ceiling.
  • Philosophical side‑threads debate whether intelligence is just pattern sampling, whether consciousness is material, and whether semantics can emerge from pure computation.

Views on current AI capabilities and limits

  • Enthusiasts: today’s models already outperform many humans at coding, some math/physics, and broad knowledge tasks; they resemble the Star Trek computer or a powerful “universal approximator.”
  • Critics:
    • LLMs are stateless, can’t truly learn on the fly, and act more like massive lookup/compression systems than “thinking” agents.
    • They struggle with intuitive multi‑constraint decisions, long‑horizon planning, and robust real‑world autonomy.
    • Multimodal generation (images/video/3D) is impressive but still often uncanny or brittle.

Economic and social impacts

  • Fears that AGI/agents will let shareholders or managers run firms with minimal employees, especially in white‑collar work.
  • Anticipated outcomes: mass unemployment, civil unrest, stronger support for socialism/redistribution, potential expropriation of AI firms, or a new techno‑feudal/serfdom scenario.
  • Others expect “abundance” and liberation from labor, but note alignment and safety are under‑discussed.

What might come after AGI

  • Common assumption: ASI via more compute, better algorithms, and swarms of coordinated agents or “AI organizations.”
  • Some speculate on AI‑run research, self‑replicating hardware (possibly off‑planet), and AI‑designed corporate structures with AI auditors and QC agents.
  • A darker camp expects human marginalization or even extinction; a few suggest a “Butlerian jihad”–style backlash as a possible response.