What can LLMs never do?

Original Article ↗ Hacker News Discussion ↗

Scope of “Never”: LLMs vs. AI in General

  • Several commenters argue the title should really be “can’t do yet”; others insist some limits are intrinsic to language models, not to AI overall.
  • Distinction drawn between “LLM” (a specific architecture trained on text) and “AGI” (a broader cognitive system with many components).

Limits of Current LLM Capabilities

  • Repeated examples: poor performance on Wordle, cellular automata (e.g., Game of Life, Rule 110), Sudoku-like tasks, simple arithmetic on long lists, counting, palindromes, and tightly specified formatting rules (e.g., number-word style, no “interesting” preface).
  • LLMs struggle with reversibility (e.g., “A is B” → “B is A”), systematic logical deduction, and stable adherence to instructions across runs.
  • Many note failures at “meta” or structural tasks: ASCII art, reasoning about grids/2D layouts, letter positions inside words, and consistency of style constraints.

Architecture, Expressivity, and AGI

  • Strong view: scaling transformers and data will not magically produce AGI; a full cognitive architecture with memory, recursion, planning, and metacognition is needed.
  • Others counter that we don’t yet know the capability limits of attention-based models; theoretical work on transformer expressivity and length generalization is cited.
  • Debate over whether universal approximation / Turing-completeness arguments meaningfully answer “never.”

Tokenization and Representation Issues

  • Many failures are attributed to subword tokenization: models “see” tokens, not characters or 2D space, so Wordle, spelling, grids, and numerical formats are hard.
  • Some argue “almost every modern problem is tokenization”; others think this is only part of deeper reasoning limits.

Prompting, Agents, and External Tools

  • Prompt engineering, chain-of-thought, retrieval-augmented generation, external memory, code execution, and agents can often patch weaknesses, but raise “who is really solving the problem?”
  • Agentic setups (loops, tools, RL fine-tuning) may extend capabilities but also risk goal drift and uncontrolled behavior.

Safety, Use, and Misuse

  • Concern about deploying LLMs for high-stakes decisions in public sector, medicine, or critical software, given unreliability and hallucinations.
  • Many see them as powerful assistive tools (coding, writing, search) but not trustworthy autonomous decision-makers.

Understanding, Reasoning, and Consciousness

  • Ongoing argument: are LLMs merely “stochastic parrots” or do they genuinely reason?
  • Lack of clear definitions of “understanding,” “reasoning,” and “AGI” makes the debate partly philosophical; several commenters highlight this ambiguity explicitly.