Markov chains are funnier than LLMs

Original Article ↗ Hacker News Discussion ↗

Perceived Humor: Markov Chains vs LLMs

  • Many report Markov bots (on IRC, Twitter, Reddit, Discord, etc.) as consistently funnier because they produce semi-coherent nonsense and abrupt context shifts.
  • Humor often comes from “unserious surprise” and the brain trying to make sense of wrong-but-plausible sentences or style mashups (e.g., Bible + programming, AWS blogs, police reports, personals, Nietzsche, Dota patch notes).
  • Several say modern LLMs feel too coherent and “clean,” so their mistakes are tedious (like a confident but ignorant uncle) rather than delightfully absurd.
  • Others argue Markov comedy is heavily cherry‑picked; most outputs are dull gibberish, whereas LLMs are broadly more usable and only less funny because of expectations.

Role of Alignment, Temperature, and Model Choice

  • Multiple commenters blame RLHF/guardrails for “sanding off” the weirdness and risk-taking that made early GPT-2/3 generations funny.
  • Suggestions: use base models, crank up temperature, disable safety prompts, or fine-tune specifically on humor to recover absurdity.
  • There’s interest in “locked” or reproducible models for research; people turn to open-weight models (Llama, WizardLM, etc.) and local runners.

Debates on Reasoning, Originality, and “Understanding”

  • One camp claims LLMs only interpolate within training data, can’t robustly reason or form new concepts, and mostly do advanced retrieval; they cite papers on failures in planning, counterfactuals, and abstraction.
  • Another camp counters that LLMs demonstrably solve many unseen problems, generalize over huge corpora, and that human reasoning is also error-prone; they argue tests often conflate “imperfect” with “no reasoning.”
  • There is disagreement over definitions of “real reasoning” and “understanding,” with some calling attempts to reserve these terms for humans circular and unfalsifiable.

Relationship Between Markov Models and LLMs

  • Several note that both are next-token predictors; LLMs can be viewed as very high-order, factored Markov models. Others push back, emphasizing transformer specifics.
  • One detailed comment argues LLMs contain the information to emulate Markov-style low-level “feel” but architecture and training bias them toward higher-level coherence, suppressing that particular flavor of absurdity.