LLMs use a surprisingly simple mechanism to retrieve some stored knowledge
Researchers probing large language models report that many factual associations inside transformers can be recovered with surprisingly simple linear functions, echoing earlier word-embedding results like “king – man + woman ≈ queen.” Commenters debate what this implies about how models store knowledge, how much they act as lossy compression of the internet, and whether hallucinations stem from failed retrieval rather than missing facts. The thread also touches on limits of current architectures, prospects for separating “reasoning” from “memory,” and whether alternative neural designs could eventually surpass transformers despite their current dominance.
Research focus: what the paper is actually claiming
- Commenters stress the work is about locating where relational facts are stored, not just verifying that the model “knows” them.
- The method fits linear functions that map internal states to specific attributes (e.g., “capital of X”), finding these work for many but not all relations.
- Some see this as modest but solid mechanistic insight; others think the press release oversells it and that results are partial and layer-limited.
Linear functions, equations, and embedding geometry
- Several comments clarify “linear function” in higher dimensions (e.g., Mv = c), not just y = mx + b.
- Discussion covers how embeddings live in high-dimensional spaces, with linear relationships supporting classic examples like “king – man + woman ≈ queen.”
- There’s debate about how “curved” internal decision surfaces really are, and why aggressive quantization still works.
Knowledge, memory, and hallucinations
- One interpretation: hallucinations can arise even when the model internally “has” a fact, if retrieval pathways fail.
- Others argue hallucinations are inseparable from the lossy compression nature of LLMs.
- Attention’s Q–K–V mechanism is discussed as a soft key–value store; some agree it’s KV-like, others see “lookup” as an overreach.
Compression, parameters, and the size of the internet
- Multiple comments frame LLMs as extreme text compressors, but clearly lossy.
- There’s back-and-forth on how much of “the internet” is actually seen in training and how much unique information exists after optimal compression.
- Clarifications distinguish parameter count from vocabulary size, and explain embeddings vs discrete tokens.
Lookup tables vs understanding and semantics
- One camp insists LLMs are sophisticated pattern matchers approximating text distributions, not semantic reasoners, and cannot ground meaning in reality.
- Another camp notes statistical models can recover useful hidden structure and that, in practice, rich learned representations can approximate semantic theories.
- A long subthread debates whether semantics can emerge from purely statistical learning or requires embodied, experimental interaction.
Architectures, local optima, and future directions
- Some argue transformers “won” after extensive experimentation; others say they may be a local maximum favored by current hardware and tooling.
- Alternatives (e.g., RNN-like or spiking models) and neuroevolution are mentioned as underexplored.
- Speculation appears on: directly writing facts into models, penalizing memorization to encourage reasoning, external memory (Paged/landmark attention), and compressing models via relation sets, but these remain open and largely speculative in the thread.