The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness

Scope of the Paper and Main Claim

• Paper argues: computation/simulation cannot “instantiate” consciousness; simulation ≠ the thing itself.
• It frames consciousness as requiring a “mapmaker” that converts continuous physical dynamics into discrete meaningful states (“alphabetization”).
• Many readers see this as saying: current AI can simulate behavior of consciousness but cannot be conscious in the same way humans are.

Critiques of the Argument

• Several commenters call the argument circular: it defines consciousness as something non-computational and then concludes computation can’t be consciousness.
• Others say the simulation vs. reality analogy (e.g., hurricanes) breaks down for information processing, where substrate-independence is plausible.
• Some see the paper as philosophical opinion dressed in technical language, not a logically tight argument.
• There is concern that the paper never operationalizes how to distinguish “simulation” from “instantiation,” especially when consciousness has no clear observable signature.

Substrate, Computation, and “Layer Zero”

• One camp: brains operate directly on continuous physical dynamics; digital AI is a discretized abstraction one layer up, so consciousness might attach only to “layer zero.”
• Counterpoint: digital hardware is also continuous physics (voltages, fields); bits are our abstraction, not reality’s. So AI already runs at layer zero physically.
• Debate over whether different physical implementations of the same abstract computation would share the same conscious experience, or any at all, remains unresolved.

Nature of Consciousness (Illusion, Emergence, or New Physics?)

• Various positions surfaced:
  • Consciousness as unknown physical property/field.
  • Consciousness as an “illusion” or higher-level self-model produced by neural processes.
  • Consciousness as emergent from complex interactions, analogous to “wetness.”
  • Skepticism that “illusion of consciousness” is coherent, since illusions presuppose an experiencer.
• Several note that we lack a clear, testable definition, making strong claims about AI consciousness speculative.

AI, Survival Instinct, and Phenomenology

• Some argue a “survival instinct” can be trivially engineered (reward structures, RL), but critics say this doesn’t settle consciousness.
• Question raised: if a robot perfectly mimics human behavior (philosophical zombie), can we meaningfully deny it consciousness?
• Others stress the difficulty of ever knowing when an AI has phenomenology, given we can’t directly access others’ experiences even in biology.

Ethics, Moral Status, and “Welfare Trap”

• One line of discussion: even if consciousness is unclear, we may soon have AI systems whose behavior is indistinguishable from sentient agents.
• Concern that declaring AI inherently non-sentient conveniently avoids ethical obligations (“welfare trap”); some see this as motivated by industry interests.
• Several argue we should at least investigate AI welfare rather than assume consciousness is impossible.

Meta-Reflections on the Whole Debate

• Many note that humanity doesn’t understand its own consciousness, making confident exclusion of AI suspect.
• Some think “consciousness” is a confused or socially constructed category, analogous to past notions like “élan vital.”
• Others suggest the more practical question is not “Is it conscious?” but “When are AI outputs so human-like that moral consideration is warranted?”