The Dunning-Kruger effect may just be a data artefact (2020)
A claim that the famous Dunning–Kruger effect may be a statistical artefact rather than a real psychological phenomenon triggers scrutiny of how the original studies were designed and analyzed. Commenters dissect issues like autocorrelation, ceiling effects, and biased simulations, while contrasting the nuanced academic result (“everyone miscalibrates, but novices do so over a wider range”) with the popular meme that “idiots don’t know they’re idiots.” Many remain convinced by their own experience that overconfident incompetence is common, but concede that psychology’s replication problems and weak methods make it hard to know how strong or special the effect really is.
Statistical critiques of the “artefact” claim
- Several commenters find the article fluffy and under-explained, especially around how the “random” simulation was generated.
- Others dig into the shared code and note issues: quartile t‑tests apparently copy‑pasted incorrectly, random lines sampled only with positive slope and bias, and an imposed positive correlation (e.g., r=0.19) that partly bakes in the effect.
- Some argue the similarity of simulated and original graphs is weak evidence: “two graphs look similar” ≠ “therefore the original result is meaningless.”
- Others point to autocorrelation and ceiling/floor effects: when self‑ratings are noisy and clamped between 0–100, low performers can only overestimate so much downward, and top performers can only overestimate so much upward, producing the characteristic “X” or “U” shapes even with noise.
Interpretations of the original Dunning–Kruger findings
- Multiple commenters stress the original result was modest: low performers thought they did somewhat better (e.g., F ≈ D), and high performers slightly underestimated (A ≈ B), not “idiots think they are geniuses.”
- Some note that later analyses report experts and novices over‑ and underestimate with similar frequency; experts just do so over a narrower range, implying better calibration.
- Others emphasize that when you average over quartiles and relative ranks, structural biases can create the graph even if individual self‑assessment is basically random.
Colloquial vs scientific DK, and related concepts
- Many see a split:
- Scientific DK (strong claim about “most” low performers) looks fragile under re‑analysis.
- Colloquial DK (“some people are wildly overconfident when incompetent”) clearly matches everyday observation.
- Related phenomena are discussed: impostor syndrome, “engineer’s disease”/ultracrepidarianism (experts overreaching into other domains), false‑consensus effects, and classic “you don’t know what you don’t know” competence stages.
Replication crisis and psychology skepticism
- Some participants use DK as another example in a broader replication crisis; they question whether much of psychology is rigorous science.
- Others push back: soft domains are inherently harder to measure; failed replications don’t automatically invalidate all effects, just weak methods.
Anecdotes, intuition, and meta‑irony
- Many insist DK “must” be real based on management and tech experience; others warn this is confirmation bias.
- There is recurring meta‑commentary that confidently misusing or “debunking” DK without understanding the statistics is itself an example of DK.