The Case That A.I. Is Thinking

Original Article ↗ Hacker News Discussion ↗

Access and meta-notes

  • Many comments focus on getting around the New Yorker paywall (archive links, Libby, Lynx).
  • Some note the author is a long-time HN participant, which colors how the piece is received but doesn’t change the arguments.

What does “thinking” even mean?

  • A recurring theme is that “thinking,” “intelligence,” “consciousness,” “sentience,” etc. are ill‑defined; people admit we lack agreed, testable definitions.
  • Several argue that debates quickly become semantic: like Dijkstra’s “can submarines swim?” – if you define “thinking” to require human-style consciousness, computers lose by definition.
  • Others say the term should track observable capabilities: if something solves problems, reasons, and adapts, calling it “thinking” is meaningful enough.

Arguments that LLMs are not thinking

  • Strong camp claiming LLMs are glorified autocomplete: probabilistic next‑token machines, closer to a huge if/else tree or database than a mind.
  • Points cited:
    • No agency or intrinsic goals; they never act without being prompted.
    • No persistent self-modification post‑training (no real learning, just context).
    • Hallucinations, fragile logic, and basic failures (classic “how many letters in ‘strawberry’”‑type tasks).
    • Same transformer trick works poorly in other domains (video, physics), suggesting the magic is in human language, not general cognition.
  • Some liken them to “stochastic parrots” or mirrors: powerful tools reflecting human text and biases, not genuine thinkers.

Arguments that LLMs are thinking (in some sense)

  • Others point to chain-of-thought traces, multi-step debugging, writing and running tests, revising assumptions, and solving novel coding/math tasks as evidence of genuine reasoning.
  • Emphasis that we didn’t “program the model,” we programmed the learning process; the internal circuits are discovered, not designed, and largely opaque even to creators.
  • Intelligence is framed by some as substrate‑independent computation; if a Turing‑complete system can emulate a human’s behavior arbitrarily well, calling it “thinking” and “sentient” is seen as reasonable.
  • Some suggest LLMs may approximate a subsystem of human cognition (pattern recognition, compression, concept mapping), without a full self-model or sustained goals.

Consciousness, sentience, and qualia

  • Long side threads on the “hard problem of consciousness,” qualia, identity of copies, brain simulations, and panpsychism.
  • Several note we cannot directly measure others’ subjective experience—human or machine—and in practice rely on self-report plus behavioral analogy.
  • Some propose graded or “fish-level” consciousness for current LLMs; others insist we’re nowhere near justifying that, and that new physics or at least new theory might be required.
  • There’s widespread acknowledgment that current science has no solid criterion to say an LLM is or isn’t conscious.

Capabilities, limits, and benchmarks

  • Participants highlight impressive performance on coding, debugging, and some reasoning puzzles, but also obvious brittleness and shallow world models.
  • Suggested “real” tests of human-like thinking include: independent frontier math research, robust ARC-style tasks, long-horizon interaction (months/years) without context collapse, and autonomous problem formulation.
  • Many expect LLMs to plateau on AGI-like metrics while remaining extremely useful “stochastic parrots” plus tools.

Architecture, learning, embodiment, and memory

  • Critical limitations identified: lack of continuous online learning, lack of embodiment and rich sensory input, no durable long-term memory integrated into the model.
  • Some see hope in agentic wrappers, tool use, external memory (RAG, vector DBs), and self-adapting or reasoning models; others see this as scaffolding around the same core autocomplete engine.
  • Comparisons are drawn to brains as pretrained, constantly fine‑tuned models tightly coupled to bodies; LLMs currently resemble a frozen policy with short-term working memory.

Ethics, personhood, and social effects

  • A few bring up pending legislation (e.g., Ohio bill against AI legal personhood) and worry about a future of “AI slaves” if we ever do create sentient systems.
  • Others stress that anthropomorphizing may be harmful or manipulative: it benefits vendors, and confuses users about reliability and moral status.
  • Some argue that even if AIs are not conscious, the way we treat them trains our habits toward living beings (e.g., being cruel to chatbots vs. tools as empathy practice).

Overall tone

  • The thread is sharply divided: one side sees current LLMs as a profound demystification of human thought; the other sees them as extremely powerful language tools plus 2022‑style hype, with “thinking” talk mostly rhetorical or philosophical rather than empirically grounded.