Detecting hallucinations in large language models using semantic entropy
Researchers are proposing “semantic entropy” as a way to flag when large language models (LLMs) are likely making up answers, by sampling multiple responses to a question and measuring how semantically consistent they are. Commenters debate whether such techniques meaningfully address safety and reliability, given that LLMs are optimized to produce plausible text rather than truth, and that they can still be confidently wrong even when answers are consistent. The thread broadens into questions about how to name these failures (“hallucination,” “confabulation,” “bullshit”), whether current models have or could have anything like intentions, and how much we should rely on LLMs for factual tasks versus treating them as creative or assistive tools whose outputs must be externally verified.
What LLM “hallucinations” are
- Many argue LLMs are “orthogonal to truth”: they optimize for plausible text, not correctness.
- Several posters prefer terms like “bullshit” (truth-indifferent output) or “confabulation” (false but fluent narratives) over “hallucination” (which suggests misperception by a mind).
- Others think “hallucination” is now a useful term of art and language drift is fine; critics counter that sloppy metaphors will mislead policymakers and the public.
Intentionality and anthropomorphism
- Long subthread on whether algorithms can “intend,” “lie,” or “care about” truth.
- One side: intentions require minds; computers are just deterministic / formal systems, so attributing mental states is a category error.
- Other side: we lack a settled theory of intentionality; under many metaphysical views (physicalism, panpsychism, some idealisms), it’s at least possible AIs could have genuine or pseudo-intentionality, so strong denials are premature.
- Several note that for everyday use, simulated intentionality (acting as if intentional) is often enough.
Semantic entropy method
- Core idea: sample multiple answers, cluster them by semantic equivalence (via another model), then compute an entropy over clusters.
- High semantic entropy ≈ model gives many divergent meanings ⇒ likely “confabulation.”
- Low entropy ≈ model consistently produces similar meanings ⇒ more grounded in training data.
- A variant decomposes answers into factoids, reformulates each as a question, and re-checks each factoid with the entropy method.
Critiques and limitations
- High agreement does not guarantee truth; a model can be confidently wrong (e.g., outdated training data, popular misconceptions).
- Entropy measures dispersion of the output distribution, not correctness of that distribution; knowing “how certain” the model is is different from knowing it’s right.
- May mislabel creative or multi-valid-answer tasks as hallucinations.
- Some see this as just another heuristic layered on top of a fundamentally non-truth-seeking system; others find it a useful partial safety tool, especially when full retraining is impossible.
Use cases and broader attitudes
- Proposed for high-stakes settings (e.g., public agencies) to suppress low-confidence answers and escalate to humans.
- Some argue we should frame LLMs as “hint providers” / improv text generators, not near-AGI or truth oracles.
- Ongoing tension between recognizing real utility (coding help, explanation, synthesis) and concern over overhype, misuse, and misplaced trust.