Data Dredging

Data dredging — also called data fishing or data snooping — is the practice of searching through large datasets for statistically significant patterns without a prior hypothesis. When you test enough variables against each other, some will correlate by pure chance. At a 5% significance level, one in twenty random comparisons will appear significant. The problem isn't looking at data; it's mistaking patterns found after the fact for genuine causal relationships. Data dredging produces "discoveries" that don't replicate because they were never real — they were artefacts of multiple testing on noisy data. The antidote is pre-registered hypotheses and out-of-sample validation.

Before analyzing data, state your hypothesis. If you're exploring, label the results as exploratory — not confirmatory. When multiple comparisons are tested, apply statistical corrections (Bonferroni, false discovery rate) or validate findings on a holdout dataset. Be especially skeptical of surprising findings that emerged from undirected analysis — the more surprising the result, the more likely it's a false positive from dredging.

Data dredging finds "significant" patterns in noisy data by testing enough variables without a prior hypothesis. The antidote is pre-registered hypotheses, multiple-testing corrections, and out-of-sample validation.