P-hacking

You're seeing it when a team runs an A/B test "just a little longer" after the first read disappoints, or checks multiple metrics until one shows significance. The experiment has been hacked to produce an answer.

You're seeing it when a pitch deck highlights a customer segment that grew 40% — after trying twelve different cuts. The impressive number is selection artefact, not signal.

You're seeing it when leadership requests "more analysis" until the data supports the preferred direction. The conclusion was fixed; the analysis was search.

Hold your team to pre-registered hypotheses. Define success metrics before launching experiments, not after. When a team presents an unexpected insight, ask how many cuts they tried. Reward honest null results over manufactured positives.

Simons built Renaissance Technologies on statistical arbitrage — and understood that most apparent patterns in financial data are noise. His team ran rigorous backtests with strict out-of-sample validation, rejecting strategies that only "worked" through data mining. The discipline was refusing to see signal where only noise existed. Founders face the same trap in smaller form every week: A/B tests nursed past their stopping rules, metrics sliced until they tell the story the board wants to hear, cohort analyses that conveniently confirm the roadmap. Simons' lesson is structural: insist on pre-registered hypotheses and out-of-sample validation before acting on any quantitative finding. The cost of p-hacking isn't a bad research paper — it's building strategy on statistical artefacts that dissolve under replication.