In the autumn of 1943, a group of American military analysts gathered around tables covered with diagrams of returning bombers. The planes that had made it back from missions over Europe were riddled with bullet holes, and the question was urgent: where should engineers add armor to protect aircraft and their crews? The answer seemed obvious. Look at where the planes had been hit. Reinforce those spots.
A statistician named Abraham Wald, working at the Statistical Research Group at Columbia University — a classified wartime research program that drew some of America's finest mathematical minds — looked at the same data and reached the opposite conclusion. The bullet holes on returning planes, he argued, told you almost nothing useful about where planes needed protection. They told you where planes could be hit and still return. The areas on returning aircraft that showed no damage were the areas demanding immediate reinforcement. Why? Because the planes that had been struck in those locations had not come back. They were sitting at the bottom of the English Channel, or scattered across occupied France in fields no analyst would ever visit.
The military had been staring at a map of survival, and mistaking it for a map of vulnerability. Wald saw the invisible fleet — the aircraft that never returned — and understood that the silence of those planes was the most important data point of all.
This is the founding story of survivorship bias: the systematic error that occurs when we draw conclusions from a sample that has, by a process we fail to account for, already been filtered to exclude failures. The planes that returned were a biased sample. Any inference about "where planes get hit" drawn from them would be an inference about planes that survived being hit — which is precisely the wrong question when your goal is to prevent planes from being shot down.
Wald's insight was formalized in his 1943 memorandum to the Statistical Research Group, later declassified and published, and it stands as one of the most elegant demonstrations of statistical reasoning applied under pressure. But the cognitive error he identified that autumn was not a wartime peculiarity. It is woven into the basic architecture of human thought, and it operates every time we reason from visible outcomes while the invisible failures dissolve into silence.
"If you only look at the survivors, you will systematically overestimate the quality of the strategy and underestimate its risks." — Nassim Nicholas Taleb, 2004
Defining the Bias and Its Relatives
Survivorship bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not, typically because the failures are no longer visible or accessible. The resulting sample — composed only of "survivors" — is not representative of the original population, and conclusions drawn from it will be systematically skewed toward whatever characteristics distinguish survivors from failures.
It is important to distinguish survivorship bias from related but distinct cognitive and statistical phenomena, because conflating them leads to imprecise diagnosis and imprecise correction.
| Bias / Heuristic | Definition | Core Mechanism | Typical Trigger |
|---|---|---|---|
| Survivorship bias | Overweighting evidence from entities that passed a filter; ignoring those that were eliminated | Missing data (failures are absent from the record) | Retrospective analysis of success stories, fund performance, historical records |
| Selection bias | A systematic distortion in sampling that makes the sample unrepresentative of the population | Non-random inclusion in the dataset | Clinical trials, surveys, any study where participation is voluntary or filtered |
| Publication bias | The tendency for positive or statistically significant results to be published at higher rates than null or negative results | Editorial and researcher choices about which findings to report | Scientific literature, meta-analyses, systematic reviews |
| Availability heuristic | Judging the likelihood of an event based on how easily examples come to mind | Memory retrieval ease, not actual frequency | Risk assessment, probability estimation, everyday decisions |
Survivorship bias is a species of selection bias — it is what happens when the selection mechanism is specifically the survival or success of an entity. Publication bias is a survivorship phenomenon applied to ideas and findings. The availability heuristic is a cognitive amplifier: it makes survivorship bias worse, because the surviving cases are, by definition, more memorable, more visible, and more cognitively available than the silent failures.
The Cognitive Science of Why We Fall For It
To understand why survivorship bias is so persistent, it is necessary to understand something about how the human mind processes — or fails to process — absent information.
The mind is an inference engine built for a world in which the most relevant information is usually visible. When a predator moved through the grassland, you could see the grass ripple. When fruit was ripe, it changed color. The physical world of the ancestral environment was overwhelmingly one in which signals were present rather than absent, and the cognitive systems that evolved to navigate it were calibrated accordingly. We are exquisitely sensitive to patterns in what we can observe. We are remarkably poor at reasoning from what we cannot observe.
The availability heuristic, first documented by Amos Tversky and Daniel Kahneman in their landmark 1973 paper in Cognitive Psychology, is the mental shortcut by which we estimate the likelihood or frequency of something based on how easily examples come to mind. Plane crashes come to mind easily, so we overestimate their frequency relative to car crashes. Celebrity entrepreneurs come to mind easily, so we overestimate what fraction of startup founders end up wealthy. The heuristic is not irrational in the deep sense — cognitive availability really is correlated with frequency in many domains. But it breaks down precisely when visibility is uncorrelated with frequency, which is exactly the situation survivorship bias creates. The surviving cases are visible. The failed cases are invisible. The availability heuristic then tells us the surviving cases are the typical cases. They are not.
Daniel Kahneman's WYSIATI framework — "What You See Is All There Is" — provides the deeper cognitive account. Kahneman describes WYSIATI as the tendency of System 1 thinking (the fast, automatic, associative mode) to build a coherent narrative from whatever information happens to be present, without generating a systematic search for missing information. The mind does not, by default, ask "what data am I not seeing?" It constructs a plausible story from the data present and experiences that story as an adequate account of reality. Applied to survivorship bias: when you see fifteen successful technology companies and study their shared characteristics, System 1 builds a confident causal story about what made them successful. It does not spontaneously generate the question "what did the ten thousand companies that failed share with these fifteen?" That question requires effortful, deliberate engagement — what Kahneman calls System 2.
There is a deeper epistemological wrinkle here. Absence of evidence is systematically harder to process than evidence of absence. If I show you a positive result — a drug that helped patients — you have something to engage with, a specific finding to evaluate. If I tell you that the negative results simply weren't reported, you must reason about an abstract void, a set of studies that occurred and produced results that then vanished. The human mind handles concrete, specific evidence poorly enough; it handles counterfactual voids even less well. This asymmetry means that even people who know about survivorship bias in the abstract will repeatedly fail to correct for it in practice, because the cognitive effort required to populate the invisible graveyard of failures is substantial and unrewarded by any obvious stimulus.
The evolutionary and cultural logic compounds the problem. Success stories are told, retold, published, celebrated, institutionalized. We build museums to victories. We write biographies of the founders who made it. We hold conferences where the survivors give keynote addresses about their strategies. The failures leave no such infrastructure of memory. The companies that went bankrupt leave no alumni associations organizing annual panels. The patients who died in early drug trials left no testimonials in the medical literature. The bombers that went down over Germany left no bullet-hole diagrams for the analysts at Columbia.
Four Case Studies in Survivorship Bias
Abraham Wald and the WWII Bombers (1943)
The full context of Wald's analysis is worth appreciating in its institutional detail. The Statistical Research Group, established in 1942, was one of several classified programs that mobilized American academics for the war effort. It included figures like Milton Friedman, W. Allen Wallis, and Leonard Jimmie Savage. Wald, a Romanian-born mathematician who had fled Europe after the Anschluss and whose family would eventually be murdered in Auschwitz, brought a formidable grounding in statistical theory to the group's practical problems.
When presented with the damage patterns on returning aircraft, Wald's formal analysis involved treating the visible damage distribution as a conditional probability: the probability that a plane was hit in location X, given that it survived. What the military needed was the unconditional probability — the probability that a hit in location X would be lethal to the aircraft. The two are related by Bayes' theorem, and Wald worked out that the areas showing least damage on surviving planes were precisely the areas where a hit was most likely to destroy an aircraft, because planes hit in those areas weren't in the sample at all. The memo he produced laid out this reasoning with mathematical precision, and the military applied it to subsequent armor decisions. The analysis was not fully declassified until decades later, and it became a textbook example of conditional probability reasoning.
The Mutual Fund Performance Myth (1990s-present)
For decades, actively managed mutual funds were marketed on the basis of their historical performance records. Investors were shown charts demonstrating that certain funds had consistently outperformed the market over five, ten, or fifteen years. The case for active management seemed, in this light, compelling.
What those charts systematically omitted was the graveyard. When a mutual fund performs poorly, the fund family does not typically trumpet its failure. It quietly merges the underperforming fund into a better-performing one, or closes it entirely. The historical record of the merged fund then reflects only the performance of the surviving entity. The poor-performing years of the closed fund disappear from the database. Studies that analyze "current" funds using historical data are therefore analyzing only the funds that survived — a sample already filtered for relative success.
Mark Carhart's influential 1997 study in the Journal of Finance demonstrated that, once survivorship bias in fund databases was corrected for, there was no evidence of persistent outperformance by actively managed funds that could not be explained by momentum and cost factors. Elton, Gruber, and Blake (1996) quantified the magnitude of the bias directly: studies using survivorship-biased fund databases were overestimating fund performance by approximately 0.9 percentage points per year — a substantial inflation when compounded over the ten-year periods typically used to market fund performance. The investors making decisions based on those records were making decisions based on a carefully edited version of history in which all the losers had been quietly erased.
Silicon Valley and the Myth of the Garage Visionary
The cultural narrative of Silicon Valley is told through its survivors. Apple, resurrected from near-bankruptcy by a returning founder. Amazon, incubated in a garage, eventually becoming the everything store. Google, born in a Stanford dorm room. These stories are told so frequently, in such rich detail, with such careful attention to the founders' early vision and their willingness to persist through adversity, that they have come to constitute a kind of origin mythology for technology entrepreneurship.
What the mythology systematically omits is the base rate. For every Apple that survived near-bankruptcy to transformation, there were thousands of technology companies that went bankrupt and did not return. The US Bureau of Labor Statistics has consistently found that approximately 20 percent of new businesses fail within their first year, roughly 45 percent within five years, and approximately 65 percent within ten years. For technology startups, which operate in more volatile conditions and are more often deliberately high-risk ventures, failure rates are generally estimated as higher still.
The founders who failed do not typically write memoirs. Venture capitalists do not frequently hold retrospectives on the investments that lost everything. The conferences celebrate the survivors, and the survivors' characteristics — their willingness to take risks, their single-minded focus, their disregard for conventional wisdom — are retroactively identified as the causes of their success. Whether those same characteristics were equally present among the failures is a question the narrative structure of Silicon Valley mythology makes very difficult to ask.
Medical Evidence and the Missing Patients
Early clinical evidence for a range of medical interventions was contaminated by survivorship bias in a form that would take decades to fully appreciate. Studies of treatment outcomes were often conducted by examining patients who had survived long enough to complete the treatment and report their results. Patients who died before the treatment concluded, or who were too ill to respond to follow-up surveys, or who had dropped out because the treatment was making them worse, were frequently excluded from the analysis — or, more precisely, were never adequately included in it.
The result was that treatments appeared more effective than they were, because the sample reporting outcomes was filtered to patients robust enough to survive to the reporting stage. This problem was documented across oncology, cardiology, and psychiatric research. The development of intention-to-treat analysis — in which all patients assigned to a treatment are included in the analysis, regardless of whether they completed it — was a methodological correction specifically designed to counteract this form of survivorship bias. The principle is straightforward: if you only count the patients who finished the treatment, you are not analyzing the treatment, you are analyzing what happens to people who are well enough to finish treatments.
Survivorship Bias Across Domains
Business strategy has been particularly susceptible to the error Wald identified. The genre of management literature that attempts to distill the habits or practices of successful companies — from Tom Peters and Robert Waterman's In Search of Excellence (1982) onward — typically selects a sample of successful firms, identifies practices common to them, and recommends those practices as causes of success. The problem, as Phil Rosenzweig argued in The Halo Effect (2007), is that this methodology cannot distinguish practices that cause success from practices that are merely correlated with survival. Many of the companies identified as exemplars in In Search of Excellence were struggling or bankrupt within a few years of publication. More fundamentally, the methodology never examines whether failed companies shared the same practices.
Self-help and success literature is survivorship bias institutionalized. The genre is built on the testimony of survivors: people who followed a system, or developed a mindset, or applied a set of principles, and achieved a successful outcome. The testimonials are genuine. The outcomes are real. What the genre structurally cannot include is testimony from the much larger population of people who followed the same systems, developed the same mindsets, applied the same principles, and did not achieve successful outcomes — because those people, lacking the success that would make their stories publishable, are absent from the books.
Military and historical analysis faces the problem in a form close to Wald's original: archives are composed of documents that survived, written by institutions that survived, describing events from the perspective of the victors who were available to describe them. The histories of empires are largely written by empires that lasted long enough to produce historians. The strategic doctrines preserved in military tradition are those that worked well enough that the armies employing them continued to exist.
Scientific research is shaped by publication bias — the systematic tendency for positive results to appear in journals at rates far exceeding their actual occurrence in research. Easterbrook et al. (1991) demonstrated in a landmark study that research with positive outcomes was significantly more likely to be published than research with null or negative outcomes, regardless of methodological quality. This means that meta-analyses and systematic reviews — the highest level of the evidence hierarchy — are themselves contaminated if their source literature is contaminated by survivorship through publication.
Investing presents the problem in perhaps its most financially consequential form. Beyond mutual fund survivorship, Barber and Odean (2000) demonstrated in a study of individual investor accounts that investors who perform poorly tend to exit the market or dramatically reduce their trading frequency, while successful investors remain active. Studies of "active investor performance" that do not track the full initial population will systematically oversample those who continued to trade — who, by that selection mechanism, are those who did not lose badly enough to stop.
An Intellectual Lineage
Wald's 1943 memorandum was the formal origin point, but the concept's broader application required subsequent intellectual labor.
Within statistics, the problem of selection-biased samples was a known technical concern, discussed under various names in sampling theory and econometrics. But the popularization of survivorship bias as a named, widely applicable cognitive error came later.
Nassim Nicholas Taleb's The Black Swan (2007) gave survivorship bias a prominent place in a broader critique of how humans reason about rare events and long-tailed distributions. Taleb's concept of silent evidence — the vast library of outcomes that never happened, plans that never succeeded, businesses that never formed — is essentially a generalization of Wald's insight. Taleb argued that the distorting effect of silent evidence was not merely a statistical inconvenience but a fundamental feature of how human beings systematically misread history: we see the published novels, not the millions of unpublished manuscripts; we see the successful traders, not the far larger population of those who made identical bets and lost; we see the cities that survived ancient wars and draw conclusions about ancient urban planning while ignoring the cities that were destroyed and left no descendant communities to be studied.
Michael Shermer, applying skeptical inquiry to popular beliefs, extended the analysis to pseudoscience and alternative medicine, pointing out that the evidential base for many alternative treatments was built almost entirely from survivorship — people for whom the treatment coincided with recovery, while those for whom it failed simply disappeared from the account.
Kahneman's WYSIATI framework, developed most fully in Thinking, Fast and Slow (2011), provided the cognitive mechanism underlying what Wald had observed empirically and Taleb had narrated historically. WYSIATI explains not just that we fail to account for absent data, but why: the mind is an engine for generating coherent narratives from available information, and it does this job so automatically and fluently that the absence of a systematic search for missing data is never experienced as a failure.
Empirical Research on the Scope of the Problem
The empirical literature on survivorship bias has demonstrated its operation across several domains with quantitative precision.
Carhart (1997), in "On Persistence in Mutual Fund Performance," published in the Journal of Finance, found that once survivorship bias was controlled for, there was no evidence that mutual funds with above-average past performance would continue to outperform. The persistence that appeared in survivorship-biased samples largely evaporated when the full population, including closed funds, was considered.
Elton, Gruber, and Blake (1996), in "Survivorship Bias and Mutual Fund Performance" in the Review of Financial Studies, directly quantified the performance inflation caused by survivorship bias in fund databases, estimating it at roughly 0.9 percentage points annually — a figure that compounds to substantial distortion over the multi-year periods used in fund marketing.
Easterbrook, Riedel, Hess, Nakayama, Califf, and Schulman (1991), in a study of research protocols submitted to a medical ethics committee and subsequently tracked for publication, found that studies with statistically significant results were published at substantially higher rates than studies with null results, regardless of the studies' methodological quality. This was an empirical demonstration of publication bias using a pre-registered cohort of studies — a design specifically chosen to avoid the selection effects inherent in studying only published research.
Barber and Odean (2000), in "Trading Is Hazardous to Your Wealth" in the Journal of Finance, analyzed 66,465 individual brokerage accounts and found that active traders systematically underperformed passive benchmarks — but also that the population of active traders was itself a survivor sample, as poor-performing investors tended to reduce trading or exit entirely.
These studies share a common methodological feature: they all required access to the full population, including those who had been filtered out of the visible record, before the bias could be measured. This is the characteristic difficulty of correcting for survivorship bias — you must specifically seek out and incorporate the data that the selection mechanism has made invisible.
The Limits of Anti-Survivorship Reasoning
Survivorship bias is a genuine epistemic hazard, but the awareness of it can be misapplied. The corrective insight — "we should not learn from survivors alone" — does not entail that learning from survivors is impossible or that case study methodology has no value.
The appropriate response to survivorship bias is not to abandon the study of successful cases, but to study them alongside failures, with explicit attention to the selection mechanism that distinguishes the two populations. The question to ask is not "what do these successful cases have in common?" but "what do these successful cases have that the comparable failed cases lack?" The latter question requires the inclusion of the failed cases in the analysis.
This is what methodologists call a control group approach applied to naturalistic settings. Rather than asking why certain mutual funds outperformed, one asks what distinguished outperforming funds from the full population of funds — including those that were closed before the analysis date. Rather than asking what Silicon Valley successes have in common, one asks what distinguishes them from the thousands of comparable ventures that attempted similar strategies and failed.
Dead pool analysis — the systematic examination of failed companies, closed funds, discontinued treatments, and abandoned projects — is the methodological corrective Wald's insight implies. It is rarely conducted, not because it is technically impossible but because failed cases generate less institutional enthusiasm and are harder to locate. The records of failed ventures are often poorly preserved, because the organizations that would have preserved them ceased to exist.
The most powerful studies in any domain are those that track a defined initial population — all funds launched in a given year, all patients enrolled in a trial — from the beginning, before any filtering has occurred. This prospective, pre-registered, intention-to-treat approach is expensive and methodologically demanding, but it is the only design that fully avoids the distortions Wald identified in that 1943 Columbia memorandum.
Seventy years later, looking at the return flight paths of surviving bombers, we are still learning to see the planes that did not come back.
References
Wald, A. (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Statistical Research Group, Columbia University. (Declassified; republished by the Center for Naval Analyses, 1980.)
Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5(2), 207-232.
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
Taleb, N. N. (2007). The Black Swan: The Impact of the Highly Improbable. Random House.
Carhart, M. M. (1997). On persistence in mutual fund performance. Journal of Finance, 52(1), 57-82.
Elton, E. J., Gruber, M. J., & Blake, C. R. (1996). Survivorship bias and mutual fund performance. Review of Financial Studies, 9(4), 1097-1120.
Barber, B. M., & Odean, T. (2000). Trading is hazardous to your wealth: The common stock investment performance of individual investors. Journal of Finance, 55(2), 773-806.
Easterbrook, P. J., Riedel, R., Hess, G., Nakayama, J. M., Califf, R. M., & Schulman, K. A. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867-872.
Rosenzweig, P. (2007). The Halo Effect: ...and the Eight Other Business Delusions That Deceive Managers. Free Press.
Peters, T. J., & Waterman, R. H. (1982). In Search of Excellence: Lessons from America's Best-Run Companies. Harper & Row.
Shermer, M. (2002). Why People Believe Weird Things: Pseudoscience, Superstition, and Other Confusions of Our Time (revised ed.). Holt Paperbacks.
Frequently Asked Questions
What is survivorship bias?
Survivorship bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not, because the failures are no longer visible or accessible. The resulting sample composed only of survivors is not representative of the original population.
What is the Abraham Wald bomber example?
During World War II, statistician Abraham Wald at Columbia University was shown damage patterns on returning bombers and asked where to add armor. His insight: the planes hit in the undamaged locations did not return and were absent from the data. He recommended reinforcing where there were no bullet holes. This is the founding example of survivorship bias reasoning.
How does survivorship bias affect mutual fund performance data?
When a mutual fund performs poorly, fund families quietly close or merge it. The closed fund disappears from historical databases. Mark Carhart (1997, Journal of Finance) found that once survivorship bias was corrected for, there was no evidence of persistent mutual fund outperformance. Elton, Gruber, and Blake (1996) estimated the inflation at 0.9 percentage points annually.
Why is survivorship bias so hard to overcome?
Because the mind builds coherent narratives from available information without searching for missing data. Kahneman calls this WYSIATI (What You See Is All There Is). Surviving cases are visible and cognitively available; failed cases are invisible. The availability heuristic treats the visible cases as typical.
What is publication bias and how is it related?
Publication bias is the tendency for positive results to be published at higher rates than null or negative results. It is survivorship bias applied to scientific findings. Easterbrook et al. (1991) demonstrated this empirically using a pre-registered cohort of research protocols.
How can survivorship bias be corrected?
Study the full original population including failures, not only the survivors. In clinical research, intention-to-treat analysis includes all patients assigned to a treatment regardless of whether they completed it. The most robust designs track a defined initial population from the start, before any filtering.
Is survivorship bias the same as selection bias?
Survivorship bias is a type of selection bias where the selection mechanism is specifically survival or success. Selection bias is the broader category covering any systematic distortion in sampling that makes the sample unrepresentative of the population.