Ask HN: Why does no one seem to care that AI gives wrong answers?
Many commenters argue that large language models are widely deployed despite frequently giving wrong or fabricated answers, because they are still useful for low‑stakes or creative tasks and often outperform ad‑ridden web search for quick information. Others see this tolerance as driven by hype, misaligned incentives, and users’ tendency to over‑trust fluent language, warning that the technology is unreliable for domains where accuracy, reasoning, or accountability matter. A recurring theme is that current AI is best treated as a fallible assistant whose output must be verified, not as an authoritative source of truth.
Perceived Problem: Wrong Answers & “Hallucinations”
- Many commenters say they do care; they’ve been “burned” enough to stop trusting LLMs for factual Q&A.
- “Hallucination” is viewed by some as PR spin for faulty output; others treat it as an inherent property of current LLMs.
- For some users, frequent, confident errors make LLMs effectively useless as answer bots, especially when even simple explanations are wrong.
- Others argue hallucination isn’t a showstopper in all contexts; it’s a risk to be managed with tests, checks, and product design.
Why Some Still Use It
- Strong adoption for low‑stakes or creative tasks: drafting emails, rewriting, summarizing, translations, basic code scaffolding, text-to-speech, word transformations.
- Many accept “90% right” if it saves time and they plan to verify or edit outputs.
- For inherently probabilistic tasks (e.g., sentiment analysis), higher accuracy than older methods is considered good enough.
- Some prefer LLMs over ad‑ridden, SEO‑polluted search, even if both can be wrong.
Limits of LLMs and Difficulty of Fixing
- Multiple comments stress that LLMs are language models, not factual databases; they produce likely next tokens, not guaranteed truths.
- Some see current architectures as intrinsically prone to overfitting, hidden failure modes, and irreducible hallucinations; they expect an asymptote, not explosive improvement.
- Others claim that with retrieval (RAG), few-shot prompting, and domain constraints, wrong answers can be treated as normal software bugs and pushed very low for narrow tasks.
- There’s disagreement whether better training or more complexity will ever make them reliably factual.
Comparison to Humans and Expectations
- Analogies to junior engineers: useful but must be supervised; critics respond that juniors learn and stop repeating the same mistakes, LLMs do not.
- Humans can say “I don’t know” or convey uncertainty; LLMs typically answer confidently regardless of reliability.
- Many note users anthropomorphize models and are misled by fluent, confident language, similar to how some fraudsters operate.
Business Incentives, Hype, and Regulation
- Commenters highlight incentives: investors and large vendors profit from shipping “good enough” AI now, betting that the “next model” will fix accuracy.
- Hype cycles (VR, blockchain, NFTs, now AI) are seen as driving deployment even where correctness is critical.
- Some predict a coming reckoning as enterprises discover AI agents and automation can’t deliver promised reliability.
- Concerns include lack of regulation, massive energy use, and a broader culture that tolerates broken, ad‑driven tech as long as “line go up.”