Measuring progress toward AGI: A cognitive framework
Google DeepMind’s new “cognitive framework” and Kaggle-based benchmarks for measuring progress toward artificial general intelligence (AGI) prompt sharp debate over what AGI should mean and how, or whether, it can be quantified. Commenters question whether checklists of abilities like perception, reasoning and social cognition actually capture human-like intelligence, noting that many humans would fail some proposed criteria while current large language models already excel at others. The thread widens into disputes over whether LLMs could ever become true AGI, the role of consciousness and intrinsic motivation, and the extent to which current AI progress is being reframed for marketing rather than reflecting a genuine path to human-level or superhuman minds.
Scope and purpose of the framework
- Many welcome having any structured benchmark to ground “are we at AGI yet?” debates.
- Others see it as thin content and marketing: a Kaggle-style leaderboard dressed up as a “cognitive framework,” plus a small prize pool.
- Some argue it’s effectively “crowdsourcing the goalposts” so companies can later claim AGI by definition.
Debates on defining and measuring AGI
- The listed cognitive skills (perception, generation, attention, learning, etc.) are seen as reasonable but incomplete or too narrow.
- Alternative taxonomies (working memory, processing speed, fluid/crystallized intelligence, pattern recognition, spatial reasoning) are proposed.
- Critics say many humans wouldn’t pass these metrics, yet are clearly generally intelligent, while current AIs excel at narrow expert tasks.
- Several note that AGI remains undefined; any claim that “LLMs will / won’t scale to AGI” is partly semantic.
What counts as intelligence?
- Ongoing argument over whether intelligence is:
- Capacity to accomplish tasks vs. capacity to originate and pursue goals.
- Distinct from knowledge, or inseparable from it.
- Some stress that intelligence exists on spectra and along multiple dimensions; others object when specific abilities (e.g., vivid imagery) become criteria that would exclude many humans.
LLM capabilities vs limitations
- Enthusiasts highlight dramatic gains: multi-thousand-line code generation, broad competence across domains, passing informal Turing tests, and impressive text synthesis.
- Skeptics emphasize:
- Lack of reliable unsupervised performance.
- Weak mid-conversation learning and physical intuition.
- Limited true novelty and invention when deprived of training data.
- Overhyped claims (replacing engineers soon, flawless legal/medical/financial use, autonomous operations).
Social cognition, alignment, and behavior
- Including social cognition as a core benchmark is controversial:
- Some see it as central for any system interacting with humans.
- Others note it conflates “navigating society effectively” with prosocial behavior; this could favor manipulative or malevolent agents.
- Several argue benchmarks should include explicit unwanted behaviors (alignment, non-harm) alongside capabilities.
Consciousness and sentience
- Multiple commenters think the missing dimension is consciousness or will: intrinsic goals, continuity of experience, and self-driven motivation.
- Others respond that:
- Consciousness vs. will are distinct.
- We can never directly verify anyone else’s consciousness, human or machine.
- Consciousness may be unnecessary—and even undesirable—for AGI used as a tool.
- Views split between materialist “emergent property” accounts and more spiritual or dualist perspectives, with no consensus.
Societal stakes and attitudes
- Some see AGI pursuit as a vanity or profit project; others predict major labor disruption even without true AGI.
- There is tension between excitement over current capabilities and exhaustion with hype, undefined terms, and unresolved safety/ethical issues.