Hypothesis Testing

August 2025

• Aug 14, 2025IdeasExperiment-Driven Projects for Better InsightsA product manager was sure social proof would lift trial signups, having read it in three marketing books. The version with customer logos converted 12 percent worse: the logos came from companies far larger than her small-business market, signaling the product was for enterprises. Shipping the obviously better version would have degraded conversion permanently. Good experiments need a falsifiable hypothesis and metrics fixed in advance.18 MIN35.0k

July 2025

• Jul 31, 2025IdeasMVP Experiments That TeachHarry's tested demand before manufacturing a single razor: a referral landing page where each signup got a link and unlocked free product at 1, 5 and 25 referrals. It ran one week and collected 100,000 email addresses, which became the launch customer list. This walks through the six parts of an experiment that can actually be wrong, then through smoke tests, concierge and Wizard-of-Oz MVPs, A/B testing mistakes, and interviews structured by The Mom Test.17 MIN27.3k
• Jul 29, 2025IdeasStartup Ideas Based on Validation PrinciplesOne founder built a 47-feature HR platform over nine months before speaking to a paying customer, and folded eighteen months later. Another interviewed 40 HR managers, found compliance reporting was the real pain, prototyped in two weeks, charged three clients $500 a month, and reached 30 customers and $15,000 MRR in six months. CB Insights puts building what nobody wants at roughly 35% of failures.17 MIN43.7k

August 2024

• Aug 11, 2024ExplainersWhat Is Statistics? The Science of Learning From DataThe Literary Digest mailed ten million ballots and got more than two million replies, predicting Alf Landon over Franklin Roosevelt by 57 to 43 percent. Its lists came from automobile registrations and telephone directories, which in Depression-era America skewed toward the affluent. It was wrong by 19 percentage points. George Gallup used a quota sample of roughly 50,000 stratified to match the electorate and called Roosevelt correctly.16 MIN11.5k