Responses from LLMs are not facts

Nature of LLM Outputs and “Facts”

• Core tension: LLM answers can contain facts, but they are not themselves a reliable source of facts.
• Several comments criticize the slogan “they just predict next words” as overly reductive; it describes the mechanism, not whether outputs are true.
• Others counter that the process matters: a result can be textually correct but epistemically tainted if produced by an unreliable method.
• Some argue LLMs are optimized for human preference and sycophancy—“plausible feel‑good slop”—rather than truth.

LLMs vs Wikipedia, Books, and Search Engines

• Wikipedia is framed as curated and verifiable: content must come from “reliable sources” and represent mainstream views proportionally.
• LLMs, by contrast, draw from an uncurated corpus; curation and explicit sourcing are seen as the key differentiators.
• Parallel is drawn to old advice “don’t cite Wikipedia”; similarly, LLMs and encyclopedias are tertiary sources that shouldn’t be primary citations.
• Some prefer LLMs to modern web search, which is seen as SEO‑polluted; others say it’s effectively the same content with different failure modes.

Citations, Hallucinations, and Tool Use

• Strong disagreement over “LLMs should just cite sources”:
  • One side: Gemini/Perplexity and others already attach links that are often useful, like a conversational search engine.
  • Other side: citations are frequently wrong, irrelevant, or wholly fabricated; models confidently quote text that doesn’t exist.
• Distinction is made between:
  • The LLM’s internal generation (no tracked provenance).
  • External tools (web search, RAG/agents) that fetch real URLs and which the model then summarizes—also fallible.
• Repeated anecdotes of invented journal issues, misrepresented documentation, fake poems and references highlight systematic unreliability.

How (and Whether) to Use LLMs

• Recommended workplace stance: using AI is fine, but the human is fully responsible for verifying code, data, and claims.
• Some see LLMs as “addictive toys” or “oracles”: useful for brainstorming, translation, and sparring when you already know the domain, but bad for learning fundamentals.
• Key risk: wrong and right answers are delivered with the same confidence; corrections often produce more polished but still wrong text.
• Many emphasize critical reading and cross‑checking with primary sources, regardless of whether information comes from AI, Wikipedia, search, or people.

Reactions to the Site and Messaging Style

• Several view the site as snarky, passive‑aggressive, and more like self‑affirmation for AI‑skeptics than effective persuasion.
• Others think the message is obvious and will not reach those who most need it; they advocate clearer norms like “don’t treat chatbot output as authoritative” and teaching deeper digital literacy instead.