Computer Science & Algorithms
Section 1
Core Idea
Statistical learning is the framework for learning from data: choose a model class, define loss, and fit by minimising empirical risk (e.g. squared error, cross-entropy). Core ideas include bias–variance tradeoff (simple models underfit, complex overfit), regularisation (penalise complexity: Ridge, Lasso), and cross-validation (hold out data to estimate generalisation). The additive rule and linear models are building blocks; modern ML extends this with flexible function classes and optimisation. The discipline is in controlling overfitting, choosing the right complexity, and interpreting what was learned—not just fitting curves.
Free playbook
Get The Business Model Playbook
58 business models, one visual page each: how the money flows, the metrics that matter, and who runs it. Free when you join the Faster Than Normal email.
Free. No spam. Unsubscribe anytime.
Section 2
How to See It
Understanding & Analyzing
You're seeing Statistical Learning when someone fits a model to data, tunes complexity (e.g. regularisation, cross-validation), and cares about performance on unseen data rather than fit on the training set alone. Any "learn from data" process that separates signal from noise is in this family.
Section 3
How to Use It
Define the target and the loss; choose model complexity appropriate to sample size. Use cross-validation or holdout to estimate generalisation; regularise to reduce overfitting. In business: treat "learning from data" as statistical—expect noise, guard against overfitting, and validate on out-of-sample or out-of-time data.
Decision filter
"Are we inferring structure from data? If yes, treat it as statistical learning: control complexity, validate on held-out data, and separate signal from noise. Avoid trusting in-sample fit as truth."
As a founder
Metrics, experiments, and pattern-seeking are forms of statistical learning. Demand holdout validation, be wary of overfitting to recent data, and use simple models first. Add complexity only when the data and problem justify it—same discipline as cross-validation and regularisation.
Section 5
Founders & Leaders
Daphne KollerCo-founder, Coursera; co-founder, Insitro; professor of ML
Koller's work spans probabilistic models, structure learning, and applying ML to biology and education. Statistical learning—fitting from data with controlled complexity and validation—is the backbone. Founders can adopt the discipline: define the target, choose complexity to match data, validate rigorously, and interpret what the model actually learned.
Section 7
Connected Models
Reinforces
Signal vs Noise
Statistical learning formalises the separation of signal from noise: the model is the signal; the residual is noise. Overfitting is fitting noise; regularisation and validation keep the model on signal.
Tension
Correlation vs Causation
Learning from data yields associations (correlations); causation requires more (design, instruments, structure). The tension: statistical learning is powerful for prediction; causal claims need additional assumptions and checks.
Leads-to
Regression to the Mean
Regression to the mean is a consequence of learning from noisy data: extreme observations tend to be partly noise, so predictions shrink toward the mean. Statistical learning makes that explicit in the model and the loss.
Section 8
One Key Quote
— Hastie, Tibshirani & Friedman, The Elements of Statistical Learning"Statistical learning is the process of estimating a function from a set of data." The art is in the choice of model class, loss, and regularisation so that the estimate generalises.
Section 11
Summary & Further Reading
Statistical learning fits models from data with controlled complexity (bias–variance, regularisation) and validation (cross-validation, holdout). Use it whenever you learn from data: define target and loss, avoid overfitting, and validate on unseen data.
01
Book
Canonical reference on regression, classification, regularisation, and cross-validation.
02
Book
Accessible treatment with applications; companion to ESL.
03