You got a null result. Will anyone publish it?
Publication bias & incentives
- Many see strong bias toward novel, positive findings, especially at elite journals; null results are rarely accepted, even when well done.
- Career incentives (tenure, funding, h-index) push researchers to prioritize “sexy” results over careful nulls or replications.
- Some argue this systemic behavior is now close to scientific misconduct; others frame it as predictable outcome of misaligned incentives rather than fraud.
Replication crisis & statistics
- With only positive outcomes published, false positives are inevitable and often unrecognized; regression to the mean then makes replications “fail.”
- Multiple commenters stress that one study is just one sample; robust knowledge requires different samples and replication.
- Misunderstandings of p-values and “statistical significance” recur; some note that insignificant results don’t prove the null, and huge samples can make trivial effects “significant.”
- Alternatives like confidence intervals, Bayesian methods, and higher significance thresholds in some fields are mentioned.
Value and risks of null results
- Nulls can bound effect sizes, correct false beliefs, and prevent wasted effort, but not all nulls are equally interesting.
- Concern: trivial or sloppy nulls could flood literature or be used strategically against rivals. Others respond that peer review and low citation impact limit this.
Who should do replications?
- Suggestions: make replication and logging of nulls a standard part of PhD training, or pre-register methods before data collection.
- Pushback: this is seen by some students as drudgery and bad for careers; others say it’s essential training but must be properly funded and recognized.
Alternative venues & formats
- Mention of journals and workshops dedicated to negative or “unsurprising” results, plus arXiv, preprints, blogs, and “living papers.”
- Objection: these outlets often carry less professional weight, so researchers deprioritize them despite personal willingness to share nulls.
Structural and process issues
- Peer review often evaluates results, not just methods; proposals for “results-blind” review and conditional acceptance based on pre-registered protocols.
- Cost and bureaucratic load of publishing, limited replication resources, and opaque reviewing further discourage null-result publication.