Markov chains are funnier than LLMs
Markov chain text generators, once used for quirky IRC and Twitter bots, are being nostalgically praised as funnier than today’s large language models because their low-level, context-free word mashups produce surprising, absurd combinations. By contrast, many find modern LLMs—especially heavily “aligned” services like ChatGPT—too coherent, cautious, and statistically average to deliver the same unserious surprise, even though they excel at staying on-topic and sounding human. Alongside examples of old bots, Reddit simulators, and mashups, commenters debate whether LLMs truly “reason” or simply imitate patterns, and how factors like model size, training data, and RLHF shape both humor and perceived intelligence.
Perceived Humor: Markov Chains vs LLMs
- Many report Markov bots (on IRC, Twitter, Reddit, Discord, etc.) as consistently funnier because they produce semi-coherent nonsense and abrupt context shifts.
- Humor often comes from “unserious surprise” and the brain trying to make sense of wrong-but-plausible sentences or style mashups (e.g., Bible + programming, AWS blogs, police reports, personals, Nietzsche, Dota patch notes).
- Several say modern LLMs feel too coherent and “clean,” so their mistakes are tedious (like a confident but ignorant uncle) rather than delightfully absurd.
- Others argue Markov comedy is heavily cherry‑picked; most outputs are dull gibberish, whereas LLMs are broadly more usable and only less funny because of expectations.
Role of Alignment, Temperature, and Model Choice
- Multiple commenters blame RLHF/guardrails for “sanding off” the weirdness and risk-taking that made early GPT-2/3 generations funny.
- Suggestions: use base models, crank up temperature, disable safety prompts, or fine-tune specifically on humor to recover absurdity.
- There’s interest in “locked” or reproducible models for research; people turn to open-weight models (Llama, WizardLM, etc.) and local runners.
Debates on Reasoning, Originality, and “Understanding”
- One camp claims LLMs only interpolate within training data, can’t robustly reason or form new concepts, and mostly do advanced retrieval; they cite papers on failures in planning, counterfactuals, and abstraction.
- Another camp counters that LLMs demonstrably solve many unseen problems, generalize over huge corpora, and that human reasoning is also error-prone; they argue tests often conflate “imperfect” with “no reasoning.”
- There is disagreement over definitions of “real reasoning” and “understanding,” with some calling attempts to reserve these terms for humans circular and unfalsifiable.
Relationship Between Markov Models and LLMs
- Several note that both are next-token predictors; LLMs can be viewed as very high-order, factored Markov models. Others push back, emphasizing transformer specifics.
- One detailed comment argues LLMs contain the information to emulate Markov-style low-level “feel” but architecture and training bias them toward higher-level coherence, suppressing that particular flavor of absurdity.