·Mathematics & Probability
Section 1
The Core Idea
Two things can move together without one causing the other. That sentence contains more decision-making power than most MBA curricula.
Correlation measures the statistical relationship between two variables — when one moves, the other tends to move in a predictable direction. Causation means one variable actually produces a change in the other through a identifiable mechanism. The gap between these two concepts is where fortunes are made and lost, where policies succeed or catastrophically backfire, and where the difference between a rigorous thinker and a confident fool becomes measurable.
The human brain is a causation-manufacturing machine. Evolution wired it that way. A rustle in the grass correlated with a predator attack often enough that our ancestors who inferred causation — and ran — survived to reproduce. The ones who paused to design a controlled experiment did not. The bias served its purpose on the savanna. In a world of complex systems, multivariate data, and trillion-dollar decisions, it's a liability that scales with the stakes.
The formal study of correlation began with Francis Galton in the 1880s. Galton — Charles Darwin's half-cousin, a polymath who also invented the weather map and fingerprint classification — noticed that tall parents tended to have children who were tall, but less extremely so. He called this "regression toward mediocrity" (now regression to the mean) and developed the concept of correlation to quantify the relationship. His protégé Karl Pearson formalised the mathematics in 1896 with the Pearson correlation coefficient, still the most widely used measure of linear association. Pearson was meticulous about the math and almost completely uncritical about its interpretation — a pattern that has persisted for 130 years.
The problem isn't the statistic. The problem is what people do with it. A correlation coefficient of 0.85 between advertising spend and revenue looks like proof that advertising drives sales. It might be. Or revenue growth might drive advertising budgets — companies spend more on marketing when they have more cash. Or a third variable — say, a booming economy — might independently drive both. Or the correlation might be entirely coincidental, an artefact of small samples and the human appetite for pattern.
Tyler Vigen's "Spurious Correlations" project catalogued hundreds of these accidents: the divorce rate in Maine correlates with per capita margarine consumption (r = 0.99). Nicolas Cage film appearances correlate with swimming pool drownings (r = 0.67). These are absurd — and that's the point. With enough variables and enough time periods, you will find correlations between anything and everything. The universe of possible variable pairs is effectively infinite. The subset that share a causal mechanism is vanishingly small.
The philosophical foundation runs deeper than statistics. David Hume argued in 1739 that causation itself is unobservable — we never see one billiard ball cause another to move; we see one event consistently followed by another and infer causation from the regularity. Hume's "problem of induction" remains unsolved in philosophy. In practice, science bypassed the philosophical impasse by developing methods — randomised controlled trials, instrumental variables, natural experiments, Bradford Hill's criteria — that don't prove causation with logical certainty but establish it with sufficient confidence to act.
Austin Bradford Hill articulated the most influential framework in 1965, proposing nine criteria for evaluating whether an observed association is likely causal: strength of association, consistency across studies, specificity, temporality (the cause must precede the effect), biological gradient (dose-response), plausibility, coherence with existing knowledge, experimental evidence, and analogy. No single criterion is sufficient. No single criterion is necessary. The framework is a structured way of asking: how many independent reasons do we have to believe this correlation reflects a real mechanism?
The practical implications are immediate and consequential. Every metric dashboard in every company displays correlations. Revenue correlates with headcount. Customer satisfaction correlates with feature count. Employee engagement correlates with free lunch quality. The question that separates useful analysis from expensive noise is always the same: is this relationship causal, and if so, in which direction? Get the direction wrong and you hire people to generate revenue that was actually driven by product-market fit. You build features to increase satisfaction that was actually driven by customer success onboarding. You invest in perks to boost engagement that was actually driven by meaningful work.
Consider the most expensive correlation-causation confusion in modern financial history. In the years preceding the 2008 crisis, credit rating agencies relied on models that assumed housing prices in different geographic regions were largely uncorrelated — because historically, they had been. National diversification, the models implied, caused portfolio risk reduction. The correlation structure was real in the historical data. The causal claim — that geographic diversity mechanically reduced default risk — was wrong. The underlying cause of the apparent decorrelation was that no national-level housing shock had occurred in the sample period. When subprime lending practices created systemic risk across all regions simultaneously, the decorrelation vanished. AAA-rated tranches built on the assumption of causal independence lost 60–80% of their value. The total cost exceeded $2 trillion in losses and triggered the deepest recession since the 1930s. The models were sophisticated. The causal reasoning was absent.
The cost of confusing correlation with causation compounds silently. Each misattributed cause generates a misallocated resource. Each misallocated resource produces a result that's slightly worse than expected. The shortfall gets explained by another spurious correlation, which generates another misallocation. The cycle is self-reinforcing, invisible from the inside, and often lethal to organisations that mistake data-richness for understanding.
