Bayesian Statistics: The three cultures

Original Article ↗ Hacker News Discussion ↗

Three Bayesian “cultures” and pragmatism

  • Thread centers on subjective vs objective vs “pragmatic” Bayes.
  • One framing: two axes – informative vs uninformative priors, and iteration vs no iteration – with most practitioners seen as iterative and using weakly informative priors.
  • Some see “pragmatic Bayes” as what people actually doing applied work use; others argue the “no iteration” positions are strawmen or only exist under specific academic incentives.
  • Critics say “pragmatic” is vague and risks masking unresolved foundational issues.

Frequentist vs Bayesian debates and practice

  • Many commenters view the “war” as overblown and emphasize using whatever works.
  • Others argue frequentist methods have been heavily misused (p‑hacking, eugenics, junk science), motivating Bayesian alternatives.
  • Counterpoint: Bayesian methods are equally abusable, especially with flexible software and complex models.
  • Several note that with genuinely uninformative priors, frequentist and Bayesian answers often align.

Priors, subjectivity, and “Bayesian hacking”

  • Priors are framed as explicit encodings of prior knowledge; you can examine sensitivity of posteriors to different priors.
  • Example of ghosts/ESP illustrates how strong priors demand extremely strong evidence.
  • Some worry priors resemble “stereotyping”; others argue all analysis is subjective, and making assumptions explicit is more honest.
  • Concern that iterating priors and models until “fit looks good” is akin to p‑hacking.

Iteration, model checking, and incentives

  • Strong disagreement over “no iteration”: some say iteration is essential; others note formal testing frameworks often assume no post‑hoc tweaking.
  • Scientific incentives (p < 0.05, publish or perish) push people to treat iteration as suspect, encouraging standardized tests instead of tailored models.
  • Suggestions: preregistration, blinding, strict train/test/validation splits, and clear separation of EDA from confirmatory analysis.

Machine learning / deep learning connections

  • Several note ML has long used Bayesian ideas (e.g., variational inference, probabilistic modeling), though much of modern ML is prediction‑driven and “algorithmic,” not data‑generation‑driven.
  • Neural nets can be treated in either Bayesian or frequentist ways; Bayesian deep learning frameworks and ELBO/variational methods are highlighted.
  • Debate whether ML aligns with “pragmatic Bayes” or is a distinct culture focused almost solely on predictive performance.

Foundations and meaning of probability

  • Extended side discussion on whether probability is well‑defined or falsifiable.
  • Responses range from formal measure‑theoretic definitions, to probability as plausibility/degree of belief, to the view that probabilities are only cleanly defined under symmetry assumptions.
  • Some argue heavy reliance on hypothesis testing has led whole fields into reproducibility crises.