Cognitive biases are popular - they fill listicles, business books, and slide decks. But not all of them are equally well-supported, and some famous effects have weakened or failed when researchers tried to replicate them. This is an evidence map: it rates the best-known biases and decision effects by how strong and replicable the evidence is, with each entry linked to a primary source.
The goal is to let you tell the difference between a robust, well-replicated finding you can build on and a famous-but-shaky effect you should cite with caution. Where the research is strong, we say so. Where it is contested or has failed to replicate, we say that plainly - because a bias map that ignores the replication crisis is itself misleading.
Why replication status matters here
Psychology went through a "replication crisis" in the 2010s: large coordinated efforts found that a substantial share of published findings did not reproduce at their original effect sizes. The Open Science Collaboration's 2015 attempt to replicate 100 psychology studies successfully reproduced well under half at full strength. Decision and social psychology were among the hardest hit.
So a responsible cognitive-bias reference can't just list effects - it has to flag which ones survived scrutiny. That is what this map does. "Robust" means the effect replicates widely and is broadly accepted; "supported with caveats" means real but bounded or sensitive to conditions; "contested / weak replication" means the original claim has been seriously challenged.
The evidence map
| Bias / effect | What it claims | Evidence status | Honest caveat | Key source |
|---|---|---|---|---|
| Loss aversion | Losses loom larger than equivalent gains in decision-making. | Robust (but debated in scope) | Well-replicated as a phenomenon; researchers debate whether it is as universal/large as once claimed, and it can attenuate in some contexts. | Kahneman & Tversky (1979) |
| Framing effects | The same choice, framed as a gain vs. a loss, changes preferences. | Robust | Widely replicated across domains; magnitude varies with how the framing is constructed. | Tversky & Kahneman (1981) |
| Anchoring | An initial number disproportionately influences subsequent numerical estimates. | Robust | One of the more reliably replicated effects; size depends on relevance/plausibility of the anchor. | Tversky & Kahneman (1974) |
| Availability heuristic | We judge probability by how easily examples come to mind. | Robust | Well-established; "ease of recall" mechanisms are nuanced and context-dependent. | Tversky & Kahneman (1973) |
| Confirmation bias | We seek, interpret, and recall information that confirms prior beliefs. | Robust | Broad, well-documented family of effects rather than a single tidy experiment. | Nickerson (1998) |
| Overconfidence / miscalibration | People are more confident in their judgments than accuracy warrants. | Robust | Reliable in calibration studies; "overconfidence" is several distinct phenomena (overestimation, overplacement, overprecision). | Moore & Healy (2008) |
| Planning fallacy | We underestimate the time/cost of our own projects despite past evidence. | Supported | Well-documented; reference-class forecasting is the evidence-based countermeasure. | Kahneman & Tversky (1979, intro of concept); Buehler et al. (1994) |
| Sunk cost fallacy | Past, unrecoverable investment irrationally drives future decisions. | Supported with caveats | Real, but effect size and moderators vary; some lab paradigms are weaker than the folk version implies. | Arkes & Blumer (1985) |
| Ego depletion | Self-control is a finite resource that "depletes" with use. | Contested / weak replication | A large multi-lab replication found little to no effect; treat the strong "willpower as fuel" claim as unsupported. | Hagger et al. (2016), Registered Replication Report |
| Social/behavioral priming (e.g. "elderly words slow walking") | Subtle cues unconsciously and strongly shape behavior. | Contested / failed replication | Several flagship priming results have failed to replicate; treat dramatic priming claims with strong skepticism. | Open Science Collaboration (2015) |
How to use this responsibly
- Robust effects (anchoring, framing, availability, confirmation, overconfidence) are safe to teach and design around, while remembering effect sizes are conditional.
- Supported / supported-with-caveats effects (planning fallacy, sunk cost, loss aversion) are real but should be described with their boundaries, not as iron laws.
- Contested effects (ego depletion, dramatic social priming) should be cited only with the replication failure noted - or not used as load-bearing evidence at all.
- The meta-lesson: the existence of a bias does not mean any specific intervention reliably "debiases" it. Debiasing evidence is its own, often weaker, literature.
Methodology and scope notes
- Evidence basis: foundational papers for each effect plus, where relevant, large replication efforts (e.g. Registered Replication Reports, Open Science Collaboration 2015). Status ratings weight replication evidence heavily.
- "Robust" is not "unlimited." Even well-replicated biases have moderators, cultural variation, and context limits. Treat ratings as evidence strength, not universal magnitude.
- Not individual advice. This summarises general findings about typical decision-makers; it is not personalised, clinical, financial, or legal guidance.
- Maintenance: updated when major new replication evidence changes a rating, not on a fixed schedule.
Sources
- Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124-1131. https://doi.org/10.1126/science.185.4157.1124
- Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2), 263-291. https://doi.org/10.2307/1914185
- Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453-458. https://doi.org/10.1126/science.7455683
- Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5(2), 207-232. https://doi.org/10.1016/0010-0285(73)90033-9
- Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220. https://doi.org/10.1037/1089-2680.2.2.175
- Moore, D. A., & Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502-517. https://doi.org/10.1037/0033-295X.115.2.502
- Buehler, R., Griffin, D., & Ross, M. (1994). Exploring the "planning fallacy." Journal of Personality and Social Psychology, 67(3), 366-381. https://doi.org/10.1037/0022-3514.67.3.366
- Arkes, H. R., & Blumer, C. (1985). The psychology of sunk cost. Organizational Behavior and Human Decision Processes, 35(1), 124-140. https://doi.org/10.1016/0749-5978(85)90049-4
- Hagger, M. S., et al. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11(4), 546-573. https://doi.org/10.1177/1745691616652873
- Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
Related reading on When Notes Fly
- Better Decisions Under Uncertainty - the decision-making guide this map supports
- Probabilistic Thinking for Better Decisions
- Common Decision Traps
- Learning Science Evidence Map - the companion evidence map
Last reviewed: June 2026. Compiled and reviewed by the WhenNotesFly editorial team against the cited primary sources and replication evidence. Corrections: editorial@whennotesfly.com � Editorial standards. Free to cite with attribution to When Notes Fly.
Frequently Asked Questions
Which cognitive biases are best supported by evidence?
Anchoring, framing effects, the availability heuristic, confirmation bias, and overconfidence/miscalibration are among the most robust and well-replicated. Loss aversion is robust as a phenomenon though its universality is debated. Planning fallacy and sunk cost are real but with caveats.
Which famous biases failed to replicate?
Ego depletion (the idea that willpower is a finite fuel) showed little to no effect in a large preregistered multi-lab replication. Several dramatic social/behavioral priming results (e.g. ‘elderly words slow walking’) have also failed to replicate. These should be cited only with the replication failure noted.
What is the replication crisis and why does it matter for bias claims?
In the 2010s, large efforts found many published psychology findings did not reproduce at their original strength - the Open Science Collaboration (2015) replicated well under half of 100 studies at full effect. Decision and social psychology were heavily affected, so any responsible cognitive-bias reference must flag which effects survived scrutiny.
Does knowing about a bias remove it?
Not reliably. The existence of a bias does not mean any particular ‘debiasing’ intervention works - debiasing is its own, often weaker, literature. Structural countermeasures (e.g. reference-class forecasting for the planning fallacy) tend to beat simply being aware of the bias.
Can I cite this cognitive bias evidence map?
Yes, free to cite with attribution to When Notes Fly. Each rating links to its primary source and, where relevant, the replication study, so you can verify the evidence and replication status directly.