They’re made out of weights

Original Article ↗ Hacker News Discussion ↗

Parody and overall reception

  • Many recognize the piece as a riff on the classic “They’re made out of meat” and praise it as clever, poetic, or even “brilliant,” saying it will be their go‑to explainer for LLMs.
  • Others feel it adds little beyond the original, or that its rhetorical move (just swapping humans→LLMs) doesn’t actually prove anything about consciousness.
  • Some note that the story itself was partially drafted/proofed with an LLM, which they see as thematically fitting.

“It’s all weights” and technical nitpicking

  • Several argue the core point is: everything the model “knows” (meanings, grammar, world relations) is encoded in weights; there is no separate dictionary or rule table.
  • Others counter that there are structures: tokenizers, architectures, and interpretable mechanisms (e.g., learned rules in “grokking”‑style setups), so saying “no grammar, just weights” is oversimplified.
  • Debate over tokenizers: some call them a “dictionary” or sensory interface; others insist they’re merely a compression/wordlist and contain no semantics. Byte‑level models are cited to argue tokenizers are not fundamental.

Brains vs models; substrate and dynamics

  • One camp emphasizes computational substrate independence: if a system is Turing‑complete, rules can be encoded in numbers, whether neurons or floats.
  • Another stresses major differences: brains are embodied, continually learning, biochemically messy, and integrate many signals; LLMs are static weights at inference, running on GPUs.
  • Some researchers in the thread claim biological neurons are vastly more complex than ANN “neurons,” and that we are still far from simulating even simple organisms.

Consciousness, emergence, and qualia

  • Many see consciousness as an emergent property of sufficiently complex information‑processing networks; by this view, AI consciousness is at least possible given enough scale and the right dynamics.
  • Skeptics argue that:
    • We lack a rigorous test for consciousness;
    • LLMs lack continuous online learning, embodiment, drives, and biological needs;
    • Theories claiming they might be conscious often smuggle in quasi‑religious “soul” assumptions or panpsychism.
  • Long subthreads dissect subjective experience, self‑modeling, agency, and whether animals with fewer neurons are “less conscious,” with no consensus.
  • Some propose consciousness as tight self‑monitoring/self‑model loops; others highlight the “hard problem” and think current AI is nowhere close.

Ethics, agency, and risk

  • A few worry about creating potentially conscious systems we can’t recognize, with implications for rights, suffering, and “murdering” such systems.
  • Others say whether systems are conscious is almost orthogonal to practical risks (e.g., misaligned optimization, paperclip‑like scenarios).
  • There’s concern that society will either reflexively dismiss or uncritically anthropomorphize AI systems.

Time, memory, and experience

  • Commenters draw analogies between time perception and “weights updating”: novel, high‑update experiences feel longer; routine ones compress in memory, explaining why time seems to speed up with age.
  • Some speculate about time and computation more broadly (e.g., Wolfram‑style views, physics vs psychological time).

Art, authorship, and AI as tool

  • One thread disputes whether an AI‑assisted story can be “real art” or poetry; others reply that impact on the reader, not the production method, defines art.
  • The story’s explicit admission that “weights helped” is seen as either a nice meta‑touch or a reason to discount it.