Greg Kroah-Hartman – Security in the LLM Age [video]

Original Article ↗ Hacker News Discussion ↗

Claims that Anthropic’s Mythos AI uncovered dozens of critical Linux kernel vulnerabilities are being re‑examined in light of reports that only a handful were real, mostly minor bugs that took about an hour of developer time to fix. Commenters argue this highlights a broader pattern of overhyping large language models’ capabilities, especially in security, where false positives are high and media and management often accept marketing at face value. The thread also raises concerns about legal risk from training data, the impact of LLM-generated noise on open source maintainers, and the importance of treating these tools as tightly supervised assistants rather than autonomous experts.

Mythos results on kernel bugs

  • Talk highlighted a breakdown of 79 reported “vulnerabilities”: many had no usable detail, some weren’t bugs at all, some used invented data, and ~15 were already fixed.
  • Around 20 needed any code changes, and only ~10 were considered “real” bugs with plausible impact; several relied on very contrived attack setups.
  • Mythos miscounted its own issues, reinforcing the perception that LLMs are brittle even on basics like counting.
  • Some commenters note Mythos did find serious bugs elsewhere (e.g., NFS, BSD/OpenBSD stacks), so it isn’t useless, but Linux “smoking guns” were absent.

Noise, false positives, and CVEs

  • Maintainers report that automated and LLM-based security reports are dominated by verbose, overconfident false positives that are expensive to triage.
  • Kernel practice is to assign CVEs to essentially every bugfix, which explains high CVE counts; Mythos‑found issues all received CVEs but were mostly low‑impact.
  • A ~50% false‑positive rate for LLM tools is contrasted with older static analyzers at ~20%, which already struggled to gain adoption.
  • Key asymmetry: bug reports are now nearly free to generate, but verification cost for experts is unchanged.

LLMs as tools: capability vs limitations

  • Many frame current frontier models as powerful pattern matchers: good at finding recurring code patterns, weak on real‑world context and threat modeling.
  • Analogies range from “overconfident intern” to “top‑percentile grad on speed,” but several argue this flatters LLMs; interns learn and take responsibility, models don’t.
  • Others counter that “pattern matching” is all humans do too, and if machines find defects humans miss, that’s valuable regardless of philosophical status.

Hype, media, and trust

  • Mythos is widely seen as overhyped: marketed as dangerously powerful, tightly access‑controlled, yet produced mostly trivial or already‑fixed issues in Linux.
  • Commenters connect this to broader AI PR (claims of “too dangerous to release”, “sentient” models, visits to the Pope) and to journalists uncritically amplifying vendor narratives.
  • Repeated six‑month “this time it’s amazing” cycles lead to growing skepticism, even among users who find LLMs practically useful.

Security implications and threat models

  • Some argue that even one real bug plus agents that can chain exploits is concerning; LLMs also shorten time from bug to working exploit.
  • Others stress that discovered kernel bugs were largely non‑critical and/or niche (e.g., malicious filesystem images, exotic networking cases).
  • Debate over future “autonomous, self‑replicating AI worms”: some see this as plausible once capable local models are common; others note current hardware and deployment constraints.

Legal and process concerns

  • One cited claim: only a minority of common training corpora are clearly legal to use; the rest is a gray zone.
  • This raises questions about incorporating LLM‑generated patches into projects like the kernel if outputs resemble copyrighted training data.
  • Existing mechanisms like Developer Certificate of Origin are seen as necessary but not wholly resolving the new risks; outcome of pending court decisions is viewed as uncertain.

Practical usage patterns and advice

  • Several recommend:
    • Treat LLMs strictly as assistive tools, not autonomous developers.
    • “Ignore the comments, look at the code” – generated explanations are often wrong; code is what matters.
    • Never upload non‑public or sensitive data to SaaS models; run open‑weights models locally when possible.
    • Expect LLM use to strain technical collaboration, as some contributors now defer to “what my model said” over human review.