Don't be fooled–LLMs don't reason
Claims that large language models don’t truly “reason” have triggered a broader look at what reasoning and intelligence actually mean for AI systems. Commenters contrast LLMs’ statistical next-token prediction with human-like deliberation, debating whether lack of explicit beliefs, persistent internal state, or self-initiated goals disqualifies them from genuine reasoning—or merely reflects a different mechanism, much like jets versus birds. The exchange highlights unresolved questions about how closely AI must mirror human cognition, the role of brute-force computation and agents, and whether current models’ impressive outputs are sophisticated pattern-matching or an early form of non-human intelligence.
Are LLMs “really” reasoning?
- One camp argues mechanism doesn’t matter: if outputs are functionally indistinguishable from reasoning, insisting “it’s not real reasoning” is unhelpful.
- Others argue LLMs are just next‑token predictors / complex calculators: powerful transformation engines, not agents with goals or deliberation.
- Some suggest LLMs may approximate stepwise reasoning via being universal function approximators that can implicitly encode state machines.
- A minority view: reasoning may not be expressible by such systems at all, even if they’re Turing complete.
Comparisons to humans
- Multiple comments note humans also:
- Confabulate chains of thought after the fact.
- Often don’t reason deeply; rely on memorization, culture, and population‑scale “brute force.”
- Fail the article’s own three criteria (explicit epistemic state, clean separation of knowledge vs processing, faithful introspective reports).
- Some argue people are wired for “linguistic pareidolia,” over‑ascribing intelligence and agency to fluent language.
- Others counter that dismissing LLM reasoning often smuggles in unexplained human specialness (e.g., “soul in the gaps”).
Mechanism, representation, and computation
- Debate over whether lack of an explicit, inspectable belief store disqualifies LLMs from reasoning, vs. beliefs simply being encoded in weights.
- Discussion of mechanistic interpretability, “global workspace”–like internal structures (e.g., J‑space), and emergent conceptual representations.
- Side thread on brains as physical/computable systems vs. more‑than‑physical; whether being simulable in binary is conceptually sufficient.
Prompting, agents, and behavior
- Several propose prompting patterns and multi‑agent setups to induce more rigorous, explicit reasoning (confidence tracking, evidence logging, red‑teaming).
- Others note LLMs don’t self‑initiate tasks or “ruminate” without being embedded in always‑on agent frameworks. Whether that’s a design choice or a deep limitation is disputed.
- Chains‑of‑thought are seen by some as genuine search over candidate narratives; others view them as stylistic gloss pasted on top of an already‑chosen answer.
Pragmatic and future‑oriented views
- Many participants downplay the taxonomy fight: LLMs are extremely useful prediction engines, often better than humans at coding/maths in practice, regardless of labels.
- Others stress limits: no robust autonomous driving, no transformative scientific breakthroughs, engineers still must supervise and correct.
- Some see the article as partly setting up a new “reasoning model” startup, but still agree that more explicit, inspectable reasoning structures would be valuable.