In 1976, Daniel Kahneman and a group of Israeli curriculum specialists were designing a new course on judgment and decision-making. The team was asked to estimate how long the project would take to complete. Everyone provided estimates. The range was 18 to 30 months. The average was 24 months.
Kahneman then turned to one member of the team — a curriculum expert with decades of experience — and asked him a different question: of all the curriculum projects he had seen that were at a similar stage, what fraction had ever been completed? The answer: he couldn't think of any that had actually been completed. How long had the ones that did complete taken? Between seven and ten years. Did he think this project was different? After reflection: not particularly.
The team's estimate remained at approximately 24 months. The project took eight years to complete — if it counted as complete at all.
Kahneman later described this as the clearest illustration of the planning fallacy he had ever seen: a group of intelligent, well-informed people systematically ignored the base rate evidence that was directly available to them, in favor of a plan-based estimate that turned out to be wildly optimistic. They were using what he called the inside view — reasoning from the specifics of their project — when the outside view — reasoning from comparable historical cases — was available and obviously more informative.
Reference class forecasting is the systematic application of the outside view. It is not a correction factor to apply to inside-view estimates. It is a method that replaces the inside view with historical base rates as the primary anchor for prediction. Its core claim is not philosophical but empirical: the statistical distribution of outcomes for comparable past projects is a better predictor of your project's outcome than any plan-based analysis you can construct.
"The outside view is implemented by identifying a reference class of similar undertakings, obtaining the statistics of the reference class, and using these statistics to generate a base-rate prediction. The inside view is set aside." — Daniel Kahneman, Thinking, Fast and Slow (2011)
Key Definitions
Reference class forecasting — A prediction method that uses the statistical distribution of outcomes for a class of comparable past cases as the basis for forecasting, rather than analyzing the specific features of the current case. Corrects for the systematic optimism of inside-view prediction by anchoring on actual historical performance.
Inside view — A prediction approach based on analyzing the specific features of the current project: breaking it into tasks, estimating each, and summing. The inside view generates optimistic estimates because it models the plan rather than the execution environment.
Outside view — A prediction approach based on asking: what actually happened in cases like this one? The outside view uses the reference class distribution as the anchor and adjusts only for factors that clearly distinguish the current case from the reference class average.
Planning fallacy — The systematic tendency to underestimate time, costs, and risks of future actions while overestimating benefits. Documented by Kahneman and Tversky (1979). Reference class forecasting is the primary empirically validated method for correcting it.
Reference class — The set of past projects or cases used as comparators for the current prediction. Choosing the right reference class — specific enough to be informative, broad enough to have statistical power — is the central methodological challenge.
Optimism bias — The tendency to believe positive outcomes are more likely and negative outcomes less likely than base rates warrant. In project planning, manifests as underestimating time, costs, and obstacles. Strong in both individuals and organizations.
Base rate — The background probability or average outcome for a category of events, based on historical experience rather than analysis of the specific case.
Distributional forecasting — An approach to prediction that produces not a single point estimate but a probability distribution over possible outcomes, capturing the full range and relative likelihoods. Reference class forecasting naturally produces distributions, not point estimates.
Infrastructure Cost Overruns: The Reference Class Data
Bent Flyvbjerg's database of over 2,000 large infrastructure projects across 70 years and 20 countries provides the most comprehensive reference class data available for major capital projects.
| Project Type | Average Cost Overrun | Percentage with Overruns | Average Schedule Overrun |
|---|---|---|---|
| Nuclear power plants | +117% | 96% | +64% |
| Large dams | +96% | 90% | +44% |
| IT projects | +73% | 75% | N/A |
| Olympics / World Cup | +156% | 100% | +n/a |
| Rail projects | +45% | 90% | +51% |
| Road projects | +20% | 60% | +38% |
| Bridges and tunnels | +34% | 72% | +23% |
| Buildings | +37% | 67% | +21% |
Source: Flyvbjerg et al., Oxford Global Projects database, 1950-2020
These figures are not anomalies from a bad era. They are stable patterns across decades and continents. The implication is clear: inside-view estimates for large projects are systematically unreliable, and the historical distribution of overruns — not the plan — should be the starting point for any honest forecast.
Why Inside-View Estimates Fail
The Scenario Problem
When you estimate how long a project will take by analyzing the project, you are constructing a scenario: a specific sequence of events, with specific steps, specific people, specific tools, specific timelines. The scenario is internally consistent and plausible. It feels like a prediction.
But a scenario is not a probability distribution. It is a single path through a vast space of possible futures. The scenarios that planners construct naturally gravitate toward best cases — or at least toward expected cases that are better than the average of the actual distribution. Unknown unknowns — the problems, delays, and complications that no one foresaw — do not appear in the scenario because they are, by definition, unforeseeable.
The result is that inside-view scenarios are systematically optimistic. They model the world as you imagine it, not as it will be.
"We knew that the record of such projects was terrible, and that most of them took much longer to complete than planned, but we felt that our project was different." — Daniel Kahneman, Thinking, Fast and Slow (2011)
The Uniqueness Illusion
Inside-view planners routinely believe their project is different from the reference class because they can see the specific features that distinguish it. These features are real — every project is unique in some dimensions. But the uniqueness of features does not guarantee uniqueness of outcomes. Most projects that ran over budget also had distinctive features that their planners believed would enable them to avoid the pattern.
This is the uniqueness illusion: the belief that one's specific situation exempts one from the base rates that apply to similar situations. It is particularly strong in high-stakes projects, complex technical endeavors, and situations where the planner has high confidence in their own analytical abilities.
The outside view combats the uniqueness illusion by forcing explicit comparison with comparable cases and by requiring justification for deviating from the reference class distribution. The burden of proof shifts: instead of assuming the current project will perform like the plan, you assume it will perform like the reference class unless specific, articulable reasons distinguish it.
Strategic Misrepresentation
Flyvbjerg identified a second mechanism beyond cognitive bias: strategic misrepresentation. Some cost and time underestimates are not accidental — they are deliberate understatements designed to make projects appear approvable that might not survive honest scrutiny. In competitive funding environments (government project bids, internal capital allocation, venture pitches), understating costs and timelines improves the chances of approval.
This means that even if cognitive biases could be fully corrected, incentive structures would still push estimates toward optimism. Reference class forecasting mitigates this partly by grounding estimates in historical data that is harder to manipulate than inside-view projections.
The Method: How Reference Class Forecasting Works
Bent Flyvbjerg describes a four-step process, developed through application to major infrastructure projects starting with the Edinburgh tram project in the early 2000s.
Step 1: Select the Reference Class
Identify past projects that are most comparable to the current one. The reference class should be:
- Relevant: Similar in type, scale, complexity, and delivery environment
- Large enough: At least 20-30 cases to produce statistically meaningful distributions
- Unbiased: Not selected to make the current project look good or bad
The reference class selection involves judgment. Classes that are too narrow may contain too few cases for statistical analysis. Classes that are too broad may include projects so different from the current one that the comparison is uninformative. When in doubt, err toward broader classes — they capture more of the true variability in outcomes.
Step 2: Establish the Reference Class Distribution
Using historical data on the reference class, construct a probability distribution of outcomes — typically for both cost and schedule, expressed as ratios of actual to planned performance. The distribution should characterize:
- The median outcome (50th percentile)
- The average outcome (mean)
- The optimistic tail (10th percentile)
- The pessimistic tail (90th percentile)
- The frequency of extreme overruns
Flyvbjerg's data on large infrastructure projects shows, for example, that the average road project overruns its cost estimate by 20%, while 60% of projects overrun by at least 10% and a substantial fraction overrun by more than 50%.
Step 3: Position the Specific Project in the Distribution
Assess where the specific project falls in the reference class distribution, using project-specific information to adjust from the reference class base rate. Factors that might justify adjusting toward more optimism or more pessimism:
More pessimistic than average:
- Higher technical novelty
- First time the organization has attempted this type of project
- Complex political or regulatory environment
- Multiple dependencies on external parties
More optimistic than average:
- Strong organizational track record with similar projects
- Exceptionally well-developed project management systems
- Simple, stable technical environment
- No regulatory dependencies
The key discipline: adjustments should be made explicitly, justified by specific evidence, and in proportion to the evidence. The default is the reference class distribution, not the inside-view estimate.
Step 4: Generate the Probabilistic Forecast
Combine the reference class distribution with any project-specific adjustments to produce a probability distribution over possible outcomes. Express this as a range — not a single point estimate — to communicate that the outcome is genuinely uncertain.
A useful output format: "Based on the reference class of comparable projects, there is a 10% probability the project completes within budget, a 50% probability it completes within 20% over budget, and a 90% probability it completes within 50% over budget."
Evidence: Does It Work?
Kahneman and Lovallo on Corporate Forecasting
Daniel Kahneman and Dan Lovallo documented the planning fallacy in corporate strategic decisions in a 2003 Harvard Business Review paper. They found that major corporate investments — mergers, acquisitions, capital projects, new product launches — consistently showed the same directional bias as infrastructure projects: overoptimistic on costs, timelines, and market reception.
Their recommendation: before approving major strategic investments, systematically search for reference classes of comparable past decisions and use the statistical record of those decisions as the base case for evaluation. They called this "taking the outside view."
"Taking the outside view involves making a forecast based on a large database of similar cases rather than on the particulars of the current situation." — Daniel Kahneman and Dan Lovallo, Harvard Business Review (2003)
Applications in Software Development
The software industry has developed reference-class-inspired methods under different names. Empirical process control — the basis of Scrum and agile methodologies — uses actual team velocity from past sprints to forecast future velocity rather than relying on bottom-up task estimates. This is reference class forecasting applied at the team level: the reference class is the same team's own recent history.
Monte Carlo simulation methods in project management sample from historical distributions of task duration variance rather than relying on single-point estimates, effectively implementing reference class forecasting probabilistically across a project's task structure.
Brooks' Law — "Adding manpower to a late software project makes it later" — is a reference-class observation: the historical record of software projects shows that adding people to late projects makes them later, not earlier. The inside view says adding more people should accelerate delivery; the outside view (Brooks' Law) correctly identifies the pattern.
UK Government and Oxford Adoption
The UK Treasury formally adopted reference class forecasting as a required element of major public project appraisal following Flyvbjerg's work. The UK's Green Book (guidance on government project appraisal) now includes explicit requirements for optimism bias adjustments based on reference class data. Denmark, the Netherlands, and Switzerland have implemented similar requirements.
Oxford's Global Projects Center, founded by Flyvbjerg, has developed standardized reference class data for major project types that is now used by governments and major infrastructure developers worldwide.
Common Mistakes in Application
Choosing a Reference Class to Justify the Plan
The most common mistake is selecting a reference class that is biased toward the inside-view estimate — finding a sample of projects that performed unusually well and using that sample as the reference class. This inverts the purpose of the method: the point is to anchor on the realistic distribution, not to find historical precedent for the desired outcome.
Using Point Estimates Instead of Distributions
Expressing the reference class forecast as a single number — "similar projects took an average of 18 months" — throws away information. The distribution is the forecast. The full range of outcomes and the probability of each region is what enables realistic planning, contingency setting, and risk communication.
Treating the Reference Class Forecast as a Target
Reference class forecasting produces a prediction about what will happen — not a target for what should happen. Using the reference class estimate as the project's budget or schedule target, rather than as a risk-adjusted range with contingencies, undermines the practical value of the method. The correct use is to set the project target inside the optimistic portion of the reference class distribution while funding a contingency reserve to cover the expected range of overruns.
Applying Only to Large Projects
Reference class forecasting applies at any scale where historical data exists. Software sprints, marketing campaigns, personal projects, learning goals, writing deadlines — all benefit from asking "what has typically happened when people attempted things like this?" rather than constructing an optimistic inside-view scenario.
Everyday Applications
The principle scales down cleanly from infrastructure megaprojects to individual decisions. Whenever you estimate how long something will take, consider:
Before constructing a plan-based estimate: Search for reference class data. How long have similar tasks taken for you in the past? How long have similar projects taken for comparable people or organizations? This historical data is your anchor.
Identify your reference class: For a writing project, the reference class might be your own previous writing projects of similar length and complexity. For learning a skill, it might be how long it has taken other people to reach a comparable skill level.
Adjust explicitly for distinguishing factors: What is genuinely different about this instance that warrants departing from the base rate? Be conservative about your adjustments — most factors you believe make your situation exceptional either do not matter as much as you think or are offset by factors you have not considered.
Communicate ranges, not point estimates: "This will take 2 weeks" communicates false precision. "This will most likely take 3-4 weeks, with a 20% chance it takes longer than 6" communicates realistic uncertainty and enables better planning.
References
- Kahneman, D., & Tversky, A. (1979). Intuitive prediction: Biases and corrective procedures. TIMS Studies in Management Science, 12, 313-327.
- Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
- Kahneman, D., & Lovallo, D. (2003). Delusions of success: How optimism undermines executives' decisions. Harvard Business Review, July 2003.
- Flyvbjerg, B. (2006). From Nobel Prize to project management: Getting risks right. Project Management Journal, 37(3), 5-15.
- Flyvbjerg, B. (2008). Curbing optimism bias and strategic misrepresentation in planning: Reference class forecasting in practice. European Planning Studies, 16(1), 3-21.
- Flyvbjerg, B., Holm, M. S., & Buhl, S. (2002). Underestimating costs in public works projects. Journal of the American Planning Association, 68(3), 279-295.
- Flyvbjerg, B. (2023). How Big Things Get Done. Crown Currency.
- Tetlock, P. E., & Gardner, D. (2015). Superforecasting: The Art and Science of Prediction. Crown Publishers.
- HM Treasury. (2022). The Green Book: Central Government Guidance on Appraisal and Evaluation. HM Treasury.
- Brooks, F. P. (1975). The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley.
Frequently Asked Questions
What is reference class forecasting?
It is a prediction method that ignores the specific plan and asks: what happened to similar projects historically? The statistical distribution of comparable past outcomes replaces inside-view estimates as the anchor for predictions.
Who developed reference class forecasting?
Daniel Kahneman and Amos Tversky developed the underlying concept in the 1970s. Bent Flyvbjerg at Oxford formalized it as a practical methodology for infrastructure planning, backed by a database of 2,000+ megaprojects.
What is the difference between the inside view and outside view?
The inside view predicts by analyzing the specific project's plan and steps. The outside view asks what actually happened in comparable cases. The inside view is systematically optimistic; the outside view corrects this by anchoring on historical base rates.
How do you choose the right reference class?
The reference class should be similar in project type, scale, complexity, and delivery environment, and large enough (20-30+ cases) to produce meaningful statistics. When in doubt, err toward broader classes rather than narrower ones.
Does reference class forecasting work in everyday decisions, not just megaprojects?
Yes. The principle applies anywhere you are estimating activities that have been done before: writing a report, launching a product, learning a skill. Start with what typically happens, then adjust for what genuinely distinguishes your situation.
What if my project is genuinely unique?
Almost no project is unique at the right level of abstraction. Find the reference class at the appropriate level of generality. If no reference class exists, that itself signals you are in genuinely uncharted territory where optimism bias will be especially pronounced.