It is time to stop teaching frequentism to non-statisticians (2012)

Preprints, Blogs, and “Cargo Cult” Science

• Some argue it’s odd that non–peer-reviewed work appears on arXiv and suggest blogs/Substack instead.
• Others reply that:
  • Preprint servers are designed for unreviewed work; using them isn’t misuse.
  • They provide DOIs and stable archiving, unlike personal blogs or platforms that may vanish.
  • Yes, arXiv has an endorsement system, so it’s not totally “anyone can post.”
• There’s concern that using a preprint server purely for the optics of credibility is “cargo cult science,” but others note formal journal publication is not required for work to be scientific.

Gatekeeping, Peer Review, and Scale

• One side: “only what is said matters; why gatekeep?”
• Counter: with billions of people publishing, credentials and peer review are crucial filters; peer review is supposed to check content and is often double blind.
• Others note that current journal systems have serious issues (replication crisis, incentives) and don’t obviously “scale better” than open models.
• Analogy: GitHub allows everyone to upload code; that doesn’t mean we treat all repos equally, but we also don’t block uploads.

Citing Unreviewed or Informal Work

• Several commenters say you should cite important preprints if they’re relevant, even if unreviewed.
• In some fields, people routinely cite key preprints that never made it to formal publication.
• Extreme example: if a correct solution to a math problem appeared on an anonymous forum, you’d still need to acknowledge it somehow.

What to Teach Non‑Statisticians

• Some think the article is old and not well argued; instead of “Frequentism vs Bayes” they’d focus first on exploratory data analysis and understanding data/phenomena.
• There’s frustration that many scientists/ML practitioners can run sophisticated methods but can’t properly inspect data, detect leakage, or match metrics to real goals.

Frequentist vs Bayesian: Competing Philosophies

• Pro‑Bayes points:

  • Frequentism treats probability as long-run frequency; Bayes treats it as degree of belief and is more general (e.g., one-off events like Saturn’s mass or a digit of π).
  • Frequentist thinking about parallel universes or infinite repetitions is seen as metaphysically awkward and not matching the questions scientists actually ask.
  • NHST is heavily criticized as answering the wrong question (P(data|H), not P(H|data)), easy to game, and central to the reproducibility mess.

• Pro‑Frequentist or skeptical‑of‑Bayes points:

  • Frequentist methods aren’t “wrong,” just less general; they can be mathematically clean, powerful, and often equivalent to Bayes with weak/flat priors.
  • Much “Bayesian” work in practice uses nearly uninformative priors, collapsing back to frequentist-like results.
  • Priors can be highly subjective and have large, poorly appreciated effects; pretending that frequentists “secretly” use priors is disputed.
  • NHST is misused more than inherently invalid; the real problem is poor understanding and design, not the entire frequentist paradigm.

Applied, Pragmatic View: Tools, Not Ideologies

• Several self-described applied statisticians see the debate as overly ideological.
• View: statistics is applied math plus a way to encode uncertainty; Bayesian and frequentist methods are just different tools with trade-offs in bias, variance, interpretability, and computation.
• Choice should depend on the task (e.g., casino-like long-run guarantees vs one-off decisions, observational sciences with many confounders, or large-scale ML where full Bayes is computationally costly).