In 2018, three researchers at MIT - Soroush Vosoughi, Deb Roy, and Sinan Aral - published a paper in Science that would become one of the most cited and most discussed findings in the study of information. The paper analyzed the spread of 126,000 news stories shared by approximately three million people on Twitter between 2006 and 2017.
It was the largest study of its kind ever conducted, covering essentially the full history of the platform through its rapid growth years.
What Vosoughi, Roy, and Aral found was unambiguous and disturbing: false news stories spread faster, farther, deeper into sharing networks, and more broadly across the platform than true stories, and this was true in every content category they examined - politics, business, entertainment, science, urban legends, and natural disasters.
The margin was not small. A false news story was 70 percent more likely to be retweeted than a true story. True stories almost never reached more than 1,000 people; the top 1 percent of false news stories routinely cascaded to between 1,000 and 100,000 people.
The depth of sharing chains - how many people away from the original source a story traveled - was longer for false news than true news.
And perhaps most importantly for those who blamed algorithmic amplification or bot activity: when the researchers removed all automated accounts from the analysis and looked only at human sharing behavior, the results were unchanged.
Humans, not machines, were driving the asymmetric spread of false information.
The finding raised a question with uncomfortable implications: what does this reveal about the human mind and its relationship to information? If true news consistently loses the competition for attention and sharing, what does that tell us about the cognitive and social mechanisms that determine what people believe and spread?
"Falsehood flies, and truth comes limping after it." - Jonathan Swift, The Examiner (1710)
Key Definitions
Misinformation - False or inaccurate information, regardless of the intent of the person spreading it. Misinformation includes genuine errors, outdated information, and false claims shared in good faith by people who believe them. Most misinformation that circulates online is spread by people who do not know it is false.
Disinformation - False information that is deliberately created and spread with the intent to deceive. State-sponsored influence operations, coordinated inauthentic behavior networks, and strategically fabricated content designed to manipulate political opinion or behavior qualify as disinformation.
Malinformation - Accurate or substantially true information deployed with the intent to cause harm - for example, selectively releasing private communications to damage a political figure, or sharing accurate personal information to enable harassment. Malinformation is not false but is weaponized.
Illusory truth effect - The phenomenon in which repeated exposure to a claim increases its perceived truthfulness, even when the claim is known to be false. First described by Hasher, Goldstein, and Toppino (1977) and extensively replicated.[1]
Motivated reasoning - Reasoning that is driven by a desired conclusion rather than by objective evaluation of evidence. People with strong motivations to reach particular conclusions process confirming evidence more readily and scrutinize disconfirming evidence more harshly.
Relevant to understanding when and why people resist corrections to false beliefs.
Inoculation theory - A psychological approach to building resistance to misinformation that prebunks false claims by exposing people to weakened examples of manipulative techniques, together with explicit refutation, before they encounter the full-strength misinformation.
Accuracy nudge - A brief intervention that draws attention to the concept of accuracy before a person makes a sharing decision, shown in experiments to improve the quality of content people choose to share.
Backfire effect - The claim that corrections sometimes strengthen false beliefs by triggering identity-protective reasoning. The original finding has not replicated consistently in large-scale studies; the current evidence suggests corrections generally work, though effects are small.
Cognitive fluency - The ease with which information is mentally processed. High fluency is experienced as a positive signal and is often unconsciously attributed to familiarity and truthfulness. The illusory truth effect appears to operate through fluency mechanisms.
The Vosoughi, Roy, and Aral Study: What It Found and Why It Matters
The 2018 Science paper by Vosoughi, Roy, and Aral deserves careful attention because it resolved a methodological problem that had plagued earlier misinformation research: scale. Previous studies of misinformation spread had been limited to specific stories, specific platforms, or short time windows.
Vosoughi and colleagues built their analysis on data from the platform's entire accessible history through 2017, covering all stories categorized by six independent fact-checking organizations as true or false.[6]
The study measured spread along four dimensions: breadth (unique users reached), depth (length of sharing chains), speed (how quickly a story was shared), and structural virality (how deeply the story penetrated social networks rather than being broadcast from a single high-follower account).
False news outperformed true news on all four dimensions.
| Dimension Measured | False News | True News |
|---|---|---|
| Likelihood of retweet | 70% higher | Baseline |
| Reach (unique users) | Up to 100,000 | Rarely above 1,000 |
| Speed to 1,500 people | ~6x faster | Slower |
| Depth (sharing chain length) | Longer | Shorter |
| Emotional responses in replies | Fear, disgust, surprise | Trust, joy, anticipation |
The novelty hypothesis - that false news spread further because it was more novel - was supported by the data: false news stories showed significantly higher novelty scores based on the semantic distance between the story and the sharing users' recent Twitter activity.
Novel information attracts attention because the brain is wired to monitor for unexpected deviations from the predicted environment.
The emotional arousal hypothesis was also supported: false political news generated higher rates of fear, disgust, and surprise in replies, while true news generated more trust, joy, and anticipation.
Emotions associated with threat and negativity - fear and disgust in particular - have well-documented effects on information processing and social transmission, activating systems associated with both avoidance and warning communication.
The study's most consequential finding for policy purposes was the human agency result: removing all suspected bots from the analysis did not change the outcomes. This directly challenged a widespread narrative that blamed the spread of misinformation on automated amplification.
While bots play a role in the information ecosystem, the asymmetric spread of false information is primarily a human phenomenon, driven by ordinary cognitive and social processes.
Why Misinformation Spreads: Competing Theories
The Motivated Reasoning Account
The dominant early theory of misinformation belief and spread was motivated reasoning: people believe and share false information because it is congenial to their pre-existing identities and beliefs.
Research by Dan Kahan at Yale Law School on "cultural cognition" showed that people's positions on politically contested empirical questions - climate change, gun control, nuclear waste storage risk - were strongly predicted by their cultural identity and group membership.
People with strong tribal identities processed evidence in ways that protected those identities, crediting evidence that supported their group's position and scrutinizing contrary evidence harshly.
On this account, partisan misinformation spreads because it tells conservatives what conservatives want to believe and liberals what liberals want to believe. Correcting it is difficult because corrections threaten identity, triggering defensive responses.
The Inattention Account
A competing theory, developed primarily by Gordon Pennycook and David Rand at MIT, argues that most misinformation spread is driven not by motivated reasoning but by inattention: people share misinformation not because they want to deceive or because the content flatters their identity, but because they are not thinking about accuracy at the moment they decide to share.
Social media sharing is a fast, low-friction activity. The dominant motivations are social - people share content to entertain, to signal membership, to express emotion, to connect with others. Accuracy is not the primary criterion people apply when deciding whether to share.
When people are prompted to think about accuracy, their sharing behavior improves.
This "accuracy nudge" finding - robust across many experiments by Pennycook, Rand, and colleagues - supports the inattention account: if people were motivated to share misinformation regardless of its accuracy, prompting them to think about accuracy would not change their behavior.
Reconciling the Two Accounts
Both accounts capture important aspects of the phenomenon. Kahan's motivated reasoning framework better explains why people resist corrections to highly identity-loaded beliefs - why evangelical Christians reject evolution despite compelling evidence, or why committed partisans maintain beliefs about their party's performance despite contrary data.
Pennycook and Rand's inattention framework better explains the everyday sharing of low-stakes misinformation - the health claims, the celebrity gossip, the political memes that do not directly threaten core identities.
The distinction matters practically: if misinformation spread is primarily motivated, structural interventions (accuracy prompts, friction, labels) will have limited effects on the most identity-relevant content. If it is primarily about inattention, structural interventions should be effective.
The evidence suggests both: accuracy nudges work for much content but show weaker effects on the most ideologically charged claims.
The Illusory Truth Effect: Repetition as a Truth Signal
One of the most important and counterintuitive findings in misinformation research is that repeating a false claim to debunk it may, under some conditions, make the claim more believable.
The illusory truth effect, established in a seminal 1977 experiment by Lynn Hasher, David Goldstein, and Thomas Toppino, documents that statements rated as false at first exposure are subsequently rated as more true when encountered again.
The mechanism appears to involve cognitive fluency: when a statement is re-encountered, the neural pathways associated with processing it are already primed, making processing easier.
The brain experiences this fluency as a positive signal - easy-to-process information feels more familiar, and familiar information feels more true.
The effect has been replicated extensively and shows several important characteristics. It operates across time delays of days to weeks. It occurs for clearly implausible statements as well as plausible ones.
And - in a finding with profound implications for debunking - it operates even when people were explicitly told at first exposure that the statement was false.
A 2018 study by Pennycook, Cannon, and Rand found that statements labeled "FALSE" at first exposure were subsequently rated as somewhat truer than statements seen for the first time, though they were still rated less true than statements labeled "TRUE." Prior exposure mattered even when the prior exposure included an explicit falsity warning.[3]
This finding does not mean debunking is counterproductive - the same study found that truth labels also increased perceived truth, and that the gap between items explicitly labeled true and items explicitly labeled false was maintained.
But it does mean that strategies centered on widely repeating a false claim for the purpose of rebutting it carry some risk of increasing the claim's perceived credibility among those who do not closely attend to the rebuttal.
Some researchers recommend "truth-forward" correction strategies that emphasize the accurate alternative without repeating the false claim extensively.
The Backfire Effect: A Correction Corrected
For several years after Nyhan and Reifler's 2010 paper in Political Behavior, "When Corrections Fail," the backfire effect was treated as a robust finding: when people are shown evidence that contradicts a politically motivated belief, some of them double down, strengthening rather than moderating the belief.[2]
The paper showed that conservative participants given accurate information about the absence of WMD in Iraq, or the consequences of tax cuts on the deficit, sometimes became more confident in their original incorrect beliefs.
The finding was widely cited, influenced communication strategy, and generated a pessimistic consensus that fact-checking was not only ineffective but potentially harmful for the most strongly held partisan beliefs.
Subsequent replication efforts substantially revised this consensus. A 2019 study by Thomas Wood and Ethan Porter, published in the American Journal of Political Science, tested corrections for 52 specific political misperceptions across multiple experiments with large samples and found that corrections reliably moved beliefs in the accurate direction.[7]
The backfire effect - belief strengthening in response to correction - appeared rarely and inconsistently across their data. When they directly replicated some of the Nyhan-Reifler conditions, they did not find backfire effects.
The current state of the evidence is roughly: corrections work in that they tend to reduce false belief among those who see them, but effects are modest, they work better for factual than for value-laden claims, and they work better for people with weaker prior commitments to the false belief.
The most important limitation of correction is not backfire but reach: the fact-check rarely reaches the same population as the false claim.
Vaccine Misinformation: A Case Study
No domain of misinformation has more clearly documentable public health consequences than vaccine safety claims. The modern anti-vaccine movement traces its origins to a 1998 paper by Andrew Wakefield and colleagues published in The Lancet, which claimed a link between the MMR vaccine and autism.
The paper was subsequently retracted by The Lancet after investigation revealed that Wakefield had financial conflicts of interest, had manipulated data, and had conducted invasive procedures on children without ethical approval.
Wakefield lost his medical license. The putative MMR-autism link has been examined in numerous large epidemiological studies and is not supported by evidence.
Despite complete scientific refutation, the Wakefield claim spread and persisted.
Studies of its diffusion on social media document the characteristic patterns of misinformation spread: initial acceleration driven by emotionally resonant content (parental fear for children's health), network clustering in communities that share skepticism of institutional medicine, and resilience to debunking because the claim resonates with identity and values (parental agency, distrust of pharmaceutical industry) rather than being merely a factual dispute.
Research by Margolin, Hannak, and Levy on vaccine misinformation spread found that the messenger significantly affected whether corrections were effective: corrections from trusted in-group sources were more effective than corrections from out-group sources or from sources associated with institutional medicine.
This finding aligns with broader research on persuasion and identity-protective cognition.
COVID-19 produced an accelerated version of the same dynamics. Multiple false claims - about the origin of the virus, the safety of vaccines, and the efficacy of unproven treatments - spread globally at unprecedented speed.
The "infodemic," as the World Health Organization termed it, was shaped by the same mechanisms that Vosoughi, Roy, and Aral had documented: novelty, emotional arousal, and the structural advantages that false information enjoys in viral sharing environments.
Prebunking and Inoculation: Building Resistance
If debunking is limited by reach and by the illusory truth effect, can people be made more resistant to misinformation before they encounter it? Inoculation theory - developed by William McGuire in the 1960s and extended to the misinformation domain by Sander van der Linden at Cambridge - suggests they can.
The inoculation approach involves exposing people to weakened examples of manipulative techniques - techniques used across many specific misinformation claims - together with explicit explanation of why those techniques are misleading.
The goal is not to address specific false claims but to build generalized resistance to the categories of manipulation that false claims employ: emotional manipulation, false dilemmas, ad hominem attacks, cherry-picking evidence, false attribution to experts, and appeals to conspiracy.
Van der Linden and colleagues developed the "Bad News" online game (badnews.nl), in which players take on the role of a misinformation producer and learn to deploy six manipulation techniques.
Participants who played the game showed reduced susceptibility to misinformation in subsequent tests, and the effect held across political groups. A browser-based "Go Viral!" game targeting COVID-19 misinformation techniques showed similar effects.
A 2022 study published in Science Advances, led by Jon Roozenbeek, tested whether short prebunking videos distributed through YouTube's advertising system could inoculate large populations against misinformation techniques.[5] The study reached approximately 1.7 million users in the European Union.
Viewers who saw the prebunking ads were better at identifying manipulative techniques in subsequent tests than control users. Effect sizes were modest - consistent with the broader literature on behavioral interventions - but the scale makes even small per-person effects meaningful in aggregate.
The prebunking approach has several advantages over traditional fact-checking: it scales horizontally (it teaches a skill rather than correcting individual claims), it is prospective rather than reactive, and its effects appear to be politically non-partisan.
Its limitations include the durability of inoculation effects over time and the challenge of reaching people who are not already somewhat skeptical.
Platform Design and Structural Interventions
The question of what platforms can do to reduce misinformation without compromising free expression has generated both substantial research and substantial controversy.
Friction and accuracy prompts: Twitter's 2020 intervention requiring users to confirm they had read an article before retweeting it reduced sharing of articles from misinformation-linked sources by about 20 percent in the week following implementation.
Multiple controlled experiments by Pennycook, Rand, and colleagues have shown that presenting a single accuracy-related question before asking people to rate sharing desirability increases the proportion of accurate content they choose to share.[4]
The inattention framework predicts this: if sharing is driven partly by failure to think about accuracy, prompting accuracy consideration improves outcomes.
Context labels and community notes: Fact-check labels and context labels attached to specific pieces of content reduce engagement with labeled content, though the effect size varies by implementation. Twitter's Community Notes (formerly Birdwatch) allows users to collaboratively annotate misleading tweets with context.
Research by Wojciech Filipkowski and others suggests rated Community Notes do reduce engagement with labeled tweets. The implied truth effect - whereby unlabeled content appears implicitly endorsed - remains a concern.
Algorithmic amplification reduction: Perhaps the most debated structural question is whether reducing algorithmic amplification of high-engagement content would reduce misinformation spread.
Because misinformation tends to be emotionally arousing and therefore high-engagement, algorithms that amplify engagement also amplify misinformation.
Reducing algorithmic amplification is not the same as removing content - it simply reduces the speed and scale at which content spreads.
Research and policy debates on this question are ongoing, with platforms reluctant to change amplification systems that drive engagement and revenue.
Structural vs. individual approaches: A recurring debate in misinformation research is whether the focus should be on changing individual cognition (through education, inoculation, accuracy prompts) or on changing the structural conditions that make misinformation profitable and widely distributed.
The individualist approach assumes people can learn to identify and resist misinformation; the structuralist approach argues that the information environment is designed to exploit human cognitive vulnerabilities and must be redesigned. Both levels of intervention have evidence supporting them; both have limits.
For more on how false information is deliberately manufactured and deployed, see why disinformation spreads and why conspiracy theories spread. For analysis of how epistemic disagreements persist even among experts, see why experts disagree.
Sources & Further Reading
- Hasher, L., Goldstein, D., & Toppino, T. (1977). Frequency and the conference of referential validity. Journal of Verbal Learning and Verbal Behavior, 16(1), 107-112. DOI: 10.1016/S0022-5371(77)80012-1
- Nyhan, B., & Reifler, J. (2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303-330. DOI: 10.1007/s11109-010-9112-2
- Pennycook, G., Cannon, T. D., & Rand, D. G. (2018). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General, 147(12), 1865-1880. DOI: 10.1037/xge0000465
- Pennycook, G., Epstein, Z., Mosleh, M., Arechar, A. A., Eckles, D., & Rand, D. G. (2021). Shifting attention to accuracy can reduce misinformation online. Nature, 592, 590-595. DOI: 10.1038/s41586-021-03344-2
- Roozenbeek, J., van der Linden, S., Goldberg, B., Rathje, S., & Lewandowsky, S. (2022). Psychological inoculation improves resilience against misinformation on social media. Science Advances, 8(34). DOI: 10.1126/sciadv.abo6254
- Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146-1151. DOI: 10.1126/science.aap9559
- Wood, T., & Porter, E. (2019). The elusive backfire effect: Mass attitudes' steadfast factual adherence. Political Behavior, 41(1), 135-163. DOI: 10.1007/s11109-018-9443-y