Bayesian Statistics: The three cultures
Bayesian statistics is being reframed into three “cultures”: subjective Bayes (strongly belief-driven priors), objective Bayes (uninformative priors aiming at neutrality), and pragmatic Bayes (iteratively refining models and priors based on fit and domain knowledge). Commenters contrast these with frequentist methods, arguing that both paradigms are often misused in practice and that problems like p‑hacking, poor understanding of p‑values, and hidden biases in priors matter more than the formal philosophy. The thread also touches on how modern machine learning often blends Bayesian ideas with predictive performance goals, raising questions about when models should explain data-generating processes versus simply optimize accuracy.
Three Bayesian “cultures” and pragmatism
- Thread centers on subjective vs objective vs “pragmatic” Bayes.
- One framing: two axes – informative vs uninformative priors, and iteration vs no iteration – with most practitioners seen as iterative and using weakly informative priors.
- Some see “pragmatic Bayes” as what people actually doing applied work use; others argue the “no iteration” positions are strawmen or only exist under specific academic incentives.
- Critics say “pragmatic” is vague and risks masking unresolved foundational issues.
Frequentist vs Bayesian debates and practice
- Many commenters view the “war” as overblown and emphasize using whatever works.
- Others argue frequentist methods have been heavily misused (p‑hacking, eugenics, junk science), motivating Bayesian alternatives.
- Counterpoint: Bayesian methods are equally abusable, especially with flexible software and complex models.
- Several note that with genuinely uninformative priors, frequentist and Bayesian answers often align.
Priors, subjectivity, and “Bayesian hacking”
- Priors are framed as explicit encodings of prior knowledge; you can examine sensitivity of posteriors to different priors.
- Example of ghosts/ESP illustrates how strong priors demand extremely strong evidence.
- Some worry priors resemble “stereotyping”; others argue all analysis is subjective, and making assumptions explicit is more honest.
- Concern that iterating priors and models until “fit looks good” is akin to p‑hacking.
Iteration, model checking, and incentives
- Strong disagreement over “no iteration”: some say iteration is essential; others note formal testing frameworks often assume no post‑hoc tweaking.
- Scientific incentives (p < 0.05, publish or perish) push people to treat iteration as suspect, encouraging standardized tests instead of tailored models.
- Suggestions: preregistration, blinding, strict train/test/validation splits, and clear separation of EDA from confirmatory analysis.
Machine learning / deep learning connections
- Several note ML has long used Bayesian ideas (e.g., variational inference, probabilistic modeling), though much of modern ML is prediction‑driven and “algorithmic,” not data‑generation‑driven.
- Neural nets can be treated in either Bayesian or frequentist ways; Bayesian deep learning frameworks and ELBO/variational methods are highlighted.
- Debate whether ML aligns with “pragmatic Bayes” or is a distinct culture focused almost solely on predictive performance.
Foundations and meaning of probability
- Extended side discussion on whether probability is well‑defined or falsifiable.
- Responses range from formal measure‑theoretic definitions, to probability as plausibility/degree of belief, to the view that probabilities are only cleanly defined under symmetry assumptions.
- Some argue heavy reliance on hypothesis testing has led whole fields into reproducibility crises.