Your dashboard shows success: conversion rate up 15%, user engagement climbing, revenue per customer increasing. Every metric green. The board presentation looks excellent. Yet three months later, the company is struggling—churn accelerating, support costs exploding, product quality complaints surging. How did all the metrics look great while the business deteriorated?
Metrics mislead not because they lie (though manipulation happens), but because they tell partial truths easily mistaken for complete pictures. A metric shows one number from one angle at one point in time, and organizations treat it as comprehensive reality. The map becomes the territory. The proxy becomes the goal. The measurement becomes divorced from what it was meant to measure.
Understanding how and why metrics mislead—through gaming, misinterpretation, misalignment, and the systemic pressures that corrupt good metrics once they become targets—is essential for using measurement effectively without being deceived.
The Core Mechanisms of Misleading
Mechanism 1: Goodhart's Law
Statement: "When a measure becomes a target, it ceases to be a good measure."
Why it happens:
- Metric chosen because it correlates with goal
- Metric becomes target
- People optimize for metric
- Correlation between metric and goal breaks down
- Metric improves while goal performance deteriorates
"Measures which appear simple are not always consistent: consider the case of a profit objective. Profit is easy to specify, but difficult to achieve: an increase in profits may be brought about by cutting R&D, maintenance, or marketing expenditure, all of which may damage long-term performance." — Charles Goodhart, economist and originator of Goodhart's Law
Example: Soviet Nail Factory
Goal: Produce useful nails
Metric: Weight of nails produced (tons)
Result:
- Factory produces huge, heavy, useless nails
- Metric (tonnage) maximized
- Goal (useful nails) sacrificed
Alternative metric: Number of nails produced
Result:
- Factory produces tiny, useless nails
- Metric (count) maximized
- Goal still sacrificed
Lesson: Optimizing a proxy metric destroys its relationship to the underlying goal.
Example: Wells Fargo Account Openings
Goal: Grow customer relationships
Metric: New accounts opened per employee
Intent: More accounts = deeper customer relationships
Result:
- Employees opened millions of unauthorized accounts
- Customers didn't want or use accounts
- Metric (accounts) soared
- Goal (real customer relationships) harmed
- Reputation destroyed, billions in fines
The metric-as-target corrupted the system.
Mechanism 2: Gaming and Manipulation
Gaming: Achieving metric targets without improving (or while degrading) actual performance.
Common gaming tactics:
| Tactic | Description | Example |
|---|---|---|
| Cherry-picking | Report only favorable data | Select best time period, exclude bad segments |
| Reclassification | Change definitions to improve numbers | Reclassify customers to hide churn |
| Threshold gaming | Bunch activity to just meet targets | Close deals at month-end to hit quota |
| Sandbagging | Delay good results to next period | Hold deals if quota already met |
| Output shifting | Hit metric by sacrificing unmeasured quality | Fast support resolution, problems unresolved |
| Measurement manipulation | Change how you measure | Adjust survey timing, question wording |
As Jerry Muller wrote in The Tyranny of Metrics, "The key is to recognize that [gaming] is not the product of a few bad individuals but of a system that creates pressures to which many conscientious people will respond—and that those pressures grow stronger the more consequential the metric becomes."
Example: British Ambulance Response Times
Target: Respond to emergencies within 8 minutes
Gaming tactics:
- Stop clock when ambulance dispatched, not when it arrives
- Send fast motorcycle paramedic first (hits 8-minute target), ambulance later
- Reclassify emergencies to less urgent categories (looser targets)
Result: Response time metrics improved, but actual emergency care quality questionable.
Example: Teacher Test Score Gaming
Metric: Student test scores
Gaming:
- Narrow curriculum to tested subjects
- Teach test-taking strategies, not deep learning
- Exclude low-performers from test day
- In extreme cases: Change student answers, give answers during test
Result: Scores rise, actual educational outcomes unclear or worse.
Mechanism 3: Partial Visibility
Metrics show what they measure and hide everything else.
The illumination problem:
- Metric illuminates measured aspect
- Makes unmeasured aspects darker by comparison
- "Drunk searching for keys under streetlight" problem
As systems theorist Russell Ackoff observed, "The one thing that all foolish plans have in common is that they are unidimensional. They focus on one variable. Improvement, however, requires the simultaneous management of many variables."
Example: Page Views
What it shows: Traffic volume
What it hides:
- Traffic quality (bots vs. humans? Engaged vs. bounce?)
- Traffic intent (ready to buy vs. random visitor?)
- Traffic outcome (converted? Got value?)
Risk: Optimize for traffic volume, get useless traffic.
Example: Employee Productivity Metrics
What it shows: Hours worked, tasks completed, output produced
What it hides:
- Quality of work
- Collaboration and helping others
- Innovation and creative thinking
- Institutional knowledge building
Risk: Optimize for measured output, destroy unmeasured but critical factors.
Mechanism 4: Misinterpretation
Metrics are misinterpreted when:
- Correlation confused with causation
- Context ignored
- Statistical significance misunderstood
- Metric definition unclear
As statistician Darrell Huff wrote in How to Lie with Statistics, "The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify."
Example: Ice Cream and Drowning
Observation: Ice cream sales correlate with drowning deaths
Misinterpretation: Ice cream causes drowning
Reality: Both caused by warm weather (confounding variable)
Lesson: Correlation ≠ causation
Example: Simpson's Paradox
Phenomenon: Trend appears in subgroups but reverses when combined.
UC Berkeley admissions (1973):
- Overall: Men admitted at higher rate than women (appears discriminatory)
- By department: Women admitted at higher rates in most departments
- Explanation: Women applied to more competitive departments
Lesson: Aggregated metrics can mislead. Segmentation reveals reality.
Example: Survivorship Bias
Phenomenon: Analyzing survivors without considering those who didn't survive.
WWII aircraft armor:
- Observe bullet holes on returning planes
- Temptation: Reinforce areas with holes
- Reality: Reinforce areas without holes (planes hit there didn't return)
Lesson: Metrics based on survivors miss critical information from non-survivors.
Mechanism 5: Metric Decay
Even good metrics degrade over time.
Decay process:
- Metric initially correlates with goal
- Metric becomes target
- People learn to game it
- Metric-goal correlation weakens
- Eventually: Metric decoupled from goal
Example: Citation Counts in Academia
Original use: Citations as proxy for research impact
Early days: Reasonable correlation
As target:
- Strategic citation networks
- Self-citation
- Citation rings (we cite each other)
- Incremental publishing (more papers = more citations)
Result: Citation counts inflated, relationship to actual impact weakened.
Types of Misleading Metrics
Type 1: Vanity Metrics
Definition: Vanity metrics that look impressive but don't correlate with meaningful outcomes.
Characteristics:
- Easy to increase artificially
- Make you feel good
- Don't inform decision-making
- Don't predict business success
Examples:
| Vanity Metric | Why It Misleads | Meaningful Alternative |
|---|---|---|
| Total page views | Doesn't mean engagement or value | Conversion rate, engagement rate |
| Social media followers | Many inactive, don't convert | Engagement rate, conversion from social |
| Registered users | Most never activate | Activated users, retained users |
| App downloads | Most never opened | Day-7 retention, activated users |
| Email list size | Many unengaged | Open rate, click rate, engaged subscribers |
Danger: Celebrate vanity metrics, miss real performance.
Type 2: Proxy Metrics
Definition: Metrics that represent something else, assumed to correlate with goals.
Problem: Proxies degrade when they become targets.
Example: Hospital Readmission Rates
Proxy for: Quality of care
Logic: Better care → fewer readmissions
Gaming:
- Extend initial hospital stays (no "readmission" if never discharged)
- Discourage readmissions (treat in ER, don't formally admit)
- Select healthier patients
Result: Readmission rates improve, actual care quality unclear.
Example: Employee Satisfaction Surveys
Proxy for: Workplace health, retention risk
Logic: Satisfied employees stay, perform better
Gaming:
- Survey timing (avoid stressful periods)
- Implicit pressure to rate highly
- Survey fatigue (only most engaged respond)
Result: Scores rise, underlying issues persist.
Type 3: Ratio Distortion
Problem: Ratios can be improved by manipulating numerator or denominator, sometimes perversely.
Example: Acceptance Rate (College Rankings)
Metric: % of applicants accepted
Desired interpretation: Selectivity indicates quality
Gaming:
- Encourage unqualified students to apply (increases applications, lowers acceptance rate)
- Reject more students
- Accept students "off waitlist" (not counted in initial acceptance rate)
Result: Acceptance rate drops, doesn't mean quality increased.
Example: Conversion Rate
Metric: Conversions / Visitors
Gaming options:
- Increase numerator: Lower prices, worse targeting (more low-value conversions)
- Decrease denominator: Reduce traffic quality filters (fewer visitors, but worse overall business)
Result: Conversion rate improves, revenue may decline.
Type 4: Threshold Effects
Problem: Behavior clusters around metric thresholds, creating distortions.
Example: Standardized Test Cutoffs
Metric: % of students scoring above threshold
Gaming:
- Focus resources on "bubble students" (just below threshold)
- Neglect high-performers (already above threshold)
- Neglect low-performers (unlikely to reach threshold)
Result: More students hit threshold, but resource allocation becomes perverse.
Example: Sales Quotas
Threshold: Monthly revenue target
Distortion:
- End-of-month scramble
- Discounts to close marginal deals
- Sandbagging (delay deals if quota met)
- Revenue pulled forward (future months suffer)
Result: Monthly target hit, but annual performance and customer relationships suffer.
Domain-Specific Misleading Examples
Software Development
Misleading metric: Lines of code written
Problem:
- Incentivizes verbosity
- Discourages refactoring (reduces lines)
- Conflates activity with value
Alternative: Features delivered and adopted, bug rates, code maintainability.
Misleading metric: Story points completed
Problem:
- Story point inflation
- Focus on volume, not value
- Gaming estimation process
Alternative: User value delivered, cycle time, customer satisfaction.
Sales
Misleading metric: Pipeline value
Problem:
- Easy to inflate by adding low-quality leads
- Doesn't account for close probability
- Creates false confidence
Alternative: Weighted pipeline (probability-adjusted), win rate, actual closed revenue.
Misleading metric: Number of calls/meetings
Problem:
- Activity, not outcome
- Incentivizes quantity over quality
- Doesn't predict revenue
Alternative: Conversion rates at each stage, deal velocity, revenue per rep.
Customer Support
Misleading metric: Tickets closed per hour
Problem:
- Incentivizes quick closure, not resolution
- Encourages closing without solving problem
- Degrades customer experience
Alternative: First-contact resolution, customer satisfaction, issue recurrence rate.
Misleading metric: Average handle time
Problem:
- Rushes complex issues
- Discourages thoroughness
- Reduces help quality
Alternative: Resolution rate, customer satisfaction, issue escalation rate.
Healthcare
Misleading metric: Patient satisfaction scores
Problem:
- Can be gamed (avoid difficult conversations, over-prescribe pain meds)
- Doesn't correlate strongly with health outcomes
- May incentivize patient appeasement over best medical practice
Alternative: Health outcomes, evidence-based care adherence, patient safety indicators.
Misleading metric: Length of stay
Problem:
- Pressure to discharge quickly
- May compromise recovery
- Readmission risk increases
Alternative: Readmission rates, recovery outcomes, patient safety, patient readiness for discharge.
Education
Misleading metric: Graduation rates
Problem:
- Pressure to pass unprepared students
- Grade inflation
- Reduced academic standards
Alternative: Actual learning assessments, post-graduation outcomes, job placement rates.
Misleading metric: Test score averages
Problem:
- Teaching to test
- Narrow curriculum
- Doesn't capture deep learning
Alternative: Critical thinking assessments, project quality, long-term learning retention.
The Measurement-Target Problem
Campbell's Law
Statement: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
Translation: Using a metric as a target corrupts it.
The Lifecycle of Metric Corruption
Stage 1: Valid Proxy
- Metric correlates with goal
- Useful for monitoring
Stage 2: Increased Attention
- Metric reported prominently
- Discussed in meetings
- Used for evaluation
Stage 3: Becoming Target
- Consequences attached to metric
- Bonuses, promotions, reputation depend on it
- Metric now high-stakes
Stage 4: Gaming Emerges
- People discover how to improve metric without improving goal
- Early gaming subtle
- Metric-goal correlation weakens
Stage 5: Institutionalized Gaming
- Gaming becomes normal practice
- "Everyone does it"
- Metric fully decoupled from goal
Stage 6: Metric Crisis
- Obvious that metric no longer represents reality
- Metric changed or abandoned
- Cycle begins again with new metric
Example: British Healthcare Waiting Times
Goal: Reduce patient wait times for treatment
Metric: % of patients treated within target time (4 hours in emergency, 18 weeks for elective surgery)
Stage-by-stage corruption:
Stage 1-2: Valid proxy
- Tracks real wait times
- Identifies problem areas
Stage 3: High stakes
- Hospital funding tied to hitting targets
- Managers' careers depend on metrics
Stage 4-5: Gaming emerges and spreads
- Ambulances wait outside ER until 4-hour window achievable
- Patients reclassified to categories with longer targets
- Elective surgeries scheduled just under 18-week deadline
- Patients "pause" on waiting list (clock stops, not counted)
Stage 6: Crisis
- Obvious gaming, public outcry
- Metric no longer trusted
- Actual care quality questionable despite hitting targets
Why Organizations Keep Using Misleading Metrics
Reason 1: Metrics Look Objective
Appeal: Numbers feel scientific, unbiased, fair
Reality: Metric choice is subjective, measurement contains biases, interpretation requires judgment
Result: False confidence in flawed metrics
Reason 2: Alternatives Are Harder
Qualitative assessment:
- Requires judgment
- Time-intensive
- Harder to scale
- Less "defensible" (no single number)
Metrics:
- Quick, scalable
- Easy to compare
- Simple to report
Result: Organizations default to metrics even when misleading, because alternatives require more effort.
Reason 3: Accountability Pressure
Managers need to demonstrate results.
Metrics provide:
- "Proof" of performance
- Comparability (vs. goals, peers, past)
- Defensibility in evaluations
Without metrics: "We improved" sounds vague
With metrics: "We improved X by 23%" sounds concrete
Problem: Even misleading metrics provide cover.
W. Edwards Deming, the quality management pioneer whose work transformed postwar Japanese manufacturing, put the organizational trap plainly: "It is wrong to suppose that if you can't measure it, you can't manage it—a costly myth."
Reason 4: Gaming Is Incremental
Gaming doesn't announce itself.
Evolution:
- Start: Slight optimization (reasonable)
- Middle: Aggressive optimization (questionable)
- End: Full gaming (clear corruption)
At each step, individuals rationalize:
- "I'm just being efficient"
- "Everyone does this"
- "The metric is the goal"
Result: Gaming normalized before anyone notices.
Reason 5: Inertia and Path Dependence
Once established:
- Historical data accumulated
- Comparisons over time matter
- Changing metric feels like admitting past measurement was wrong
- Political cost to change
Result: Broken metrics persist long after problems obvious.
Detecting Misleading Metrics
Red Flag 1: Metric Improves, Reality Doesn't
Test: Does improving the metric correspond to actual goal achievement?
Example:
- Customer satisfaction scores rising
- Yet churn increasing, complaints up
- Red flag: Metric decoupled from reality
Red Flag 2: Everyone Hits Targets Easily
If targets consistently achieved:
- Targets too easy, OR
- Widespread gaming
Healthy: Some hit targets, some miss (indicates stretch goals and honesty)
Suspicious: Everyone always hits targets (indicates gaming or sandbaggin)
Red Flag 3: Unmeasured Aspects Deteriorating
If measured areas improve while unmeasured areas degrade:
- Tunnel vision
- Resources shifted from unmeasured to measured
Example:
- Metric: Feature velocity (features shipped per sprint)
- Reality: Code quality declining, technical debt rising, bugs increasing
Red Flag 4: Metric Behavior Clusters Around Thresholds
If results bunch just above threshold:
- Indicates gaming to hit target
- Natural distributions don't cluster at arbitrary thresholds
Example: Test scores clustering just above passing threshold suggests teaching narrowly to threshold.
Red Flag 5: People Can't Explain How Metric Connects to Goal
Ask: "How does improving this metric advance our actual goals?"
If answers are:
- Vague
- Circular ("We measure it because it's important")
- Inconsistent across people
Red flag: Metric has become ritualized without clear purpose.
Preventing Metric Misleading
Strategy 1: Measure Outcomes, Not Just Proxies
Closer to actual goal = harder to game.
Hierarchy:
- Worst: Activity metrics (calls made, features shipped)
- Better: Output metrics (deals closed, features adopted)
- Best: Outcome metrics (revenue, customer retention, mission impact)
Strategy 2: Use Multiple Complementary Metrics
Single metrics get gamed. Balanced scorecards resist gaming.
Example: Balanced customer support metrics
- Speed (response time)
- Quality (customer satisfaction)
- Effectiveness (first-contact resolution)
- Efficiency (cost per ticket)
Can't optimize all simultaneously without real improvement.
Strategy 3: Include Qualitative Assessment
Don't rely on metrics alone.
Balanced approach:
- Metrics for scale, trends, patterns
- Qual (conversations, observations, stories) for context, gaming detection, meaning
Strategy 4: Separate Measurement from Evaluation
**When metrics used for:
- Learning: Honest reporting, problem-solving
- Punishment: Gaming, hiding problems
Approach:
- Measure for learning and improvement (formative)
- Supplement with periodic evaluation (summative) that's harder to game
Strategy 5: Rotate Metrics
If metric becomes corrupted:
- Change or retire it
- Introduce new metric
- Forces people to refocus on goal, not metric
Strategy 6: Audit for Gaming
Regularly check:
- Are there suspicious patterns? (clustering at thresholds, sudden changes)
- Do metric improvements correspond to real outcomes?
- What are people doing to hit metrics?
If gaming detected, address root causes (incentives, consequences), not just symptoms.
Conclusion: Metrics as Tools, Not Truth
Metrics are not reality. They are models of reality—simplified, partial, distorted.
The map is not the territory.
Metrics mislead when:
- They become targets (Goodhart's Law)
- People game them
- They're misinterpreted
- They show partial picture (hide important factors)
- They decay over time as gaming evolves
Despite risks, metrics are useful:
- Enable scale (can't qualitatively assess millions)
- Identify patterns
- Track trends
- Focus attention
The path forward:
- Use metrics (don't abandon measurement)
- Don't trust metrics blindly (supplement with qualitative understanding)
- Measure outcomes (not just proxies)
- Use multiple metrics (resist gaming)
- Monitor for corruption (metrics degrade over time)
- Remember the goal (metric is tool, not objective)
Good measurement requires:
- Humility (metrics are flawed tools)
- Vigilance (watch for gaming and distortion)
- Balance (metrics + qualitative understanding)
- Purpose (remember why you're measuring)
"What gets measured gets managed"—sometimes in ways that help, often in ways that hurt.
Measure thoughtfully. Interpret carefully. Act wisely.
What Research Shows About Metric Misleading
The literature on metric dysfunction is extensive, but several researchers have produced the most influential analyses of why and how metrics mislead.
Charles Goodhart identified the most fundamental mechanism. Working as an economist at the Bank of England in the 1970s, Goodhart observed that every monetary aggregate the Bank used as a policy target lost its predictive validity once it became a target. The intuition: metrics are chosen because they correlate with outcomes. But correlation depends on people not optimizing for the metric. Once they do, the behavior that sustained the correlation changes, and the metric decouples from the outcome it was supposed to track. This is not a failure of measurement design -- it is a structural property of any proxy metric in an adversarial context. The 2016 Wells Fargo scandal, in which employees opened 3.5 million unauthorized accounts to hit cross-selling metrics, is the canonical modern business example. The metric (accounts per customer) correlated with genuine customer relationship depth when employees were trying to build relationships. It decoupled completely when they were trying to hit a number.
Donald Campbell's 1979 paper "Assessing the Impact of Planned Social Change" extended Goodhart's insight to social policy. Campbell's Law: the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures. Campbell was drawing on his own research evaluating Great Society programs in the 1960s, where he observed that programs evaluated on narrow quantitative metrics consistently gamed those metrics while the broader social problems they targeted remained unaddressed. Campbell's contribution was to identify the institutional mechanism: when a metric determines funding, careers, and reputation, entire organizations align toward improving the metric. This is not individual moral failure but rational collective adaptation to incentive structures.
Jerry Muller's The Tyranny of Metrics (2018) is the most comprehensive empirical survey of metric misleading across sectors. Muller, a historian at Catholic University of America, examined education, healthcare, policing, universities, the military, and business. His core finding is that metric fixation -- the institutional substitution of quantitative metrics for substantive judgment -- reliably produces the same set of dysfunctions: gaming, tunnel vision (unmeasured factors neglected), short-termism, risk aversion, and the demotivation of intrinsically motivated professionals. Muller's historical perspective adds an important dimension: these patterns are not new. They appeared in British colonial administration, in Soviet industrial planning, and in early 20th-century American scientific management. Metric fixation is a recurring institutional pathology, not a product of digital-age data abundance.
W. Edwards Deming approached the problem from the perspective of manufacturing quality. His critique of "management by objective" and numerical targets was grounded in statistical thinking: most variation in outcomes is produced by system factors, not individual performance. When organizations use metrics to evaluate and rank individuals, they misattribute system-level variation to individual performance, creating perverse incentives and destroying the psychological safety necessary for genuine improvement. Deming's approach was to use metrics as tools for understanding systems -- tracking variation over time to identify special causes and common causes -- rather than as targets or performance evaluations. His work transforming Japanese manufacturing quality after World War II demonstrated that this approach could produce sustained improvement that metric-based management typically could not.
Robert Kaplan and David Norton, developing the Balanced Scorecard at Harvard Business School in the early 1990s, documented a specific form of metric misleading: the exclusive focus on financial metrics in corporate performance management. Their research showed that companies relying solely on financial KPIs consistently underinvested in the drivers of future performance -- employee learning, internal process quality, customer relationships. The financial metrics looked fine until the underlying factors deteriorated sufficiently to show up in the numbers, typically 18 to 24 months later. By then, the damage was expensive to reverse. The Balanced Scorecard was designed as a multi-perspective measurement system that would reveal deterioration in leading indicators before it became visible in financial results.
Real-World Case Studies in Metric Misleading
Soviet nail production. The Soviet planned economy provides the most documented historical examples of metric gaming at industrial scale. When central planners set quotas for nail production measured in tons, factories shifted to manufacturing large, heavy nails that maximized tonnage while minimizing usefulness. When planners switched to quota by unit count, factories shifted to tiny nails. When steel sheet quotas were measured by weight, factories produced thick sheets; when measured by area, they produced thin sheets that tore in use. These examples are not apocryphal -- they are drawn from documented Soviet economic analyses and represent a predictable consequence of using proxy metrics in a system with no market feedback. The lesson is not unique to communist economies: any system that ties strong incentives to proxy metrics while eliminating market feedback mechanisms will produce similar results.
The UK waiting time targets. The Blair government introduced waiting time targets for the National Health Service in the early 2000s as a transparency and accountability measure. Patients were waiting unacceptably long for both emergency and elective treatment. The targets were genuine improvements: the 4-hour emergency department limit and 18-week elective surgery limit addressed real patient harm. As Gerry Bevan and Christopher Hood documented in their 2006 analysis in Public Administration, the targets created measurable improvement in reported wait times -- and equally measurable gaming. Emergency departments developed practices for holding ambulances outside until staff were confident the patient could be processed within the 4-hour window from formal arrival. Elective surgery waiting lists were "administratively paused" -- patients remained waiting but were temporarily removed from the counted list. Patients were reclassified into clinical categories with longer permitted wait times. The metric improved while the patient experience it was supposed to represent was manipulated in ways that did not necessarily improve care.
Enron's revenue metrics. Enron's collapse in 2001 is partly a story about how accounting metrics can be structured to mislead. The company used mark-to-market accounting to record the full expected lifetime value of long-term contracts as revenue in the current period. This produced revenue growth numbers (from $13 billion in 1996 to $101 billion in 2000) that made Enron appear to be one of the most successful companies in American history. The metric was not technically fraudulent at first -- mark-to-market was an accepted accounting method. But it created a systematic decoupling between reported revenue and cash generation. Investors and analysts who relied on the revenue metric without understanding its calculation methodology were misled not by false numbers but by a metric whose relationship to economic value had been severed. When the underlying contracts failed to generate the projected cash flows, the house of cards collapsed.
UK policing and crime statistics. British police forces, under pressure from government targets to reduce crime, have been repeatedly documented manipulating the crime statistics used to measure their performance. The methods are well-understood: downgrading offenses to less serious categories (reclassification), failing to record crimes reported by victims (no-criming), discouraging victims from making formal reports. A 2014 report by the UK Statistics Authority removed its quality mark from Home Office crime statistics, citing concerns about data integrity. Her Majesty's Inspectorate of Constabulary investigations found substantial evidence of under-recording across multiple forces. The metric (recorded crime rate) had become a performance target; the response was to manage the metric rather than crime itself. The British Crime Survey -- an independent victimization survey that asks citizens about crimes experienced regardless of whether they were reported to police -- consistently shows different trends from police-recorded statistics, revealing the gap between the metric and the reality it was supposed to represent.
Academic citation inflation. The h-index, developed by physicist Jorge Hirsch in 2005 as a metric for research impact, became widely used in academic hiring and tenure decisions within years of its introduction. The predictable consequence documented by researchers including Ludo Waltman and Nees Jan van Eck: citation practices changed. Citation rings -- groups of researchers who agree to cite each other -- became more common. Self-citation rates increased. Researchers published more shorter papers (maximizing citable units) rather than fewer comprehensive works. Journals competed for citations by publishing more review articles (heavily cited) and fewer replication studies (rarely cited). The h-index was a reasonable proxy for research impact when it was a passive measurement. As it became a high-stakes target, behavior adapted to improve the metric rather than the research quality it was supposed to represent.
Evidence-Based Principles for Resisting Metric Misleading
Principle 1: Treat all metrics as hypotheses that require validation. A metric is a claim that this number tracks that outcome. This claim may be true initially and false later, as gaming develops. Organizations should test the relationship between metrics and outcomes systematically and regularly. If a metric improves but the outcome it is supposed to predict does not improve, the metric has been corrupted or was never valid. This requires tracking both the metric and the outcome it is supposed to represent, which means maintaining some form of gold-standard outcome measurement even when it is expensive.
Principle 2: Separate measurement from evaluation. Deming's insight was that using metrics for performance evaluation creates exactly the pressure that drives gaming and corruption. Metrics used for learning and system improvement -- where the purpose is to understand what is happening, not to evaluate and rank people -- generate different behavior. Organizations that use the same metrics for both operational learning and individual performance evaluation typically end up with both functions compromised: the metrics get gamed (undermining operational learning) and the evaluation becomes disconnected from actual performance.
Principle 3: Use metrics that require value delivery to improve. The most gaming-resistant metrics are those structurally tied to genuine value creation. Customer retention can be improved by actually retaining customers; it cannot be improved by reclassifying churned customers as active. Revenue per customer can be improved by delivering more value to existing customers; it cannot be improved by opening unauthorized accounts. Metrics that can be improved through accounting manipulations, reclassifications, or procedural gaming are more susceptible to the Goodhart dynamics. Metrics that require actual behavior change in service of genuine customer or organizational value are more robust.
Principle 4: Maintain qualitative assessment alongside quantitative measurement. Muller's historical analysis shows that metric misleading is most severe in organizations that have eliminated qualitative judgment in favor of pure quantification. The reason is structural: gaming a number is much easier than gaming an experienced human observer who is asking substantive questions. Healthcare organizations that combine outcome metrics with clinical peer review, educational institutions that combine test scores with portfolio assessment, and businesses that combine performance metrics with customer relationship data are consistently more resistant to Goodhart dynamics than those relying solely on quantitative measurement.
References
Goodhart, C. (1975). "Problems of Monetary Management: The U.K. Experience." Papers in Monetary Economics (Reserve Bank of Australia).
Campbell, D. T. (1979). "Assessing the Impact of Planned Social Change." Evaluation and Program Planning, 2(1), 67–90.
Muller, J. Z. (2018). The Tyranny of Metrics. Princeton University Press.
Kerr, S. (1975). "On the Folly of Rewarding A, While Hoping for B." Academy of Management Journal, 18(4), 769–783.
Ridgway, V. F. (1956). "Dysfunctional Consequences of Performance Measurements." Administrative Science Quarterly, 1(2), 240–247.
Austin, R. D. (1996). Measuring and Managing Performance in Organizations. Dorset House.
Strathern, M. (1997). "'Improving Ratings': Audit in the British University System." European Review, 5(3), 305–321.
Power, M. (1997). The Audit Society: Rituals of Verification. Oxford University Press.
Levitt, S. D., & Dubner, S. J. (2005). Freakonomics: A Rogue Economist Explores the Hidden Side of Everything. William Morrow.
Bevan, G., & Hood, C. (2006). "What's Measured Is What Matters: Targets and Gaming in the English Public Health Care System." Public Administration, 84(3), 517–538.
de Bruijn, H. (2007). Managing Performance in the Public Sector (2nd ed.). Routledge.
Hood, C. (2006). "Gaming in Targetworld: The Targets Approach to Managing British Public Services." Public Administration Review, 66(4), 515–521.
Kahneman, D., & Tversky, A. (1973). "On the Psychology of Prediction." Psychological Review, 80(4), 237–251.
Croll, A., & Yoskovitz, B. (2013). Lean Analytics: Use Data to Build a Better Startup Faster. O'Reilly Media.
Seddon, J. (2008). Systems Thinking in the Public Sector: The Failure of the Reform Regime...and a Manifesto for a Better Way. Triarchy Press.
About This Series: This article is part of a larger exploration of measurement, metrics, and evaluation. For related concepts, see [Goodhart's Law Breaks Metrics], [Why Measurement Changes Behavior], [Vanity Metrics vs Meaningful Metrics], and [Designing Useful Measurement Systems].
Frequently Asked Questions
Why do metrics often mislead?
People game them, they get disconnected from underlying goals, they're misinterpreted, or they measure proxies poorly correlated with what matters.
What is Goodhart's Law?
When a measure becomes a target, it ceases to be a good measure—people optimize for the metric rather than the underlying goal.
How do people game metrics?
By hitting targets without improving actual performance—optimizing metric appearance while degrading what metrics were meant to measure.
What are vanity metrics?
Metrics that look impressive but don't correlate with meaningful outcomes or business goals—they make you feel good but aren't useful.
Can good metrics go bad?
Yes. Once metrics become targets with consequences, behavior shifts to optimize the metric, often degrading actual performance.
What causes metric misinterpretation?
Confusing correlation with causation, ignoring context, cherry-picking time periods, or not understanding what metrics actually measure.
How do you prevent metrics from misleading?
Use multiple metrics together, understand limitations, monitor for gaming, maintain qualitative understanding, and tie metrics to actual outcomes.
Should you ever stop measuring something?
Yes. When metrics get gamed beyond usefulness, create perverse incentives, or when behavior optimizes metrics over real goals.