Computer Science & Algorithms
Section 1
Core Idea
UCB (Upper Confidence Bound) is a simple policy for the multi-armed bandit: for each arm, compute an index = estimated mean + confidence bound (e.g. proportional to √(log n / n_i), where n is total pulls, n_i pulls of arm i). Play the arm with the highest index. The bound encodes uncertainty: underplayed arms get a higher bonus, so they get tried; as n_i grows, the bound shrinks and the policy exploits the best arm. No tuning of exploration rate—the bound is derived from concentration inequalities. Near-optimal regret in many settings and a workhorse for A/B tests, recommendation, and adaptive allocation. Sub-entry under explore–exploit.
Free playbook
Get The Business Model Playbook
58 business models, one visual page each: how the money flows, the metrics that matter, and who runs it. Free when you join the Faster Than Normal email.
Free. No spam. Unsubscribe anytime.
Section 2
How to See It
Deciding & Judging
You're seeing Upper Confidence Bound when a system allocates traffic or effort to options (variants, channels, items) by ranking them on "estimate plus uncertainty bonus"—favour options that are either good or under-tested. The formula is often mean + c√(log N / n_i).
Section 3
How to Use It
Maintain a running mean and count per arm; at each step compute UCB = mean + c√(ln N / n_i) (c a constant, N total pulls, n_i pulls of arm i). Play the arm with max UCB. Works for bandits, A/B/n tests, and any setting where you learn by trying. Prefer UCB when you want a simple, parameter-light policy with good theoretical guarantees.
Decision filter
"Are we repeatedly choosing among options and learning payoffs as we go? If yes, UCB gives a principled explore–exploit balance: try under-sampled options (high bound) and exploit high-mean options (high mean + shrinking bound)."
As a founder
Allocating traffic to variants, channels, or features is a bandit. UCB: score each option by observed performance plus an uncertainty bonus, then send traffic to the top. No manual exploration rate—the bound handles it. Use it for automated A/B tests and adaptive funnels.
Section 5
Founders & Leaders
Ken GriffinFounder & CEO, Citadel
Griffin built Citadel on quantitative allocation and adaptive strategies—systematic explore–exploit across instruments and signals. UCB is the same idea in bandit form: one index per option (expected value + uncertainty), allocate to the highest. Founders can use UCB for product and growth experiments: automate the balance between trying new things and doubling down on what works.
Section 7
Connected Models
Reinforces
Explore-exploit Tradeoff
UCB is a concrete resolution of the tradeoff: the confidence bound is the exploration term; the mean is the exploit term. Together they form one index per arm; the policy is greedy in that index.
Tension
The Gittens Index
Gittins is the optimal index for the discounted bandit; UCB is a simple, computable approximation. The tension: UCB is easier and works well; Gittins is exact but heavier. Use UCB in practice; think Gittins for the principle.
Leads-to
Signal vs Noise
UCB separates signal (the mean) from noise (the bound): with few samples, the bound is large and the arm gets tried; with many, the mean dominates. Learning is reducing the noise term over time.
Section 8
One Key Quote
— Auer, Cesa-Bianchi & Fischer, Finite-time Analysis of the Multiarmed Bandit Problem (2002)"UCB1 selects the arm that maximizes the sum of the empirical mean and an upper confidence bound." The bound is chosen so that the true mean lies below it with high probability; the policy is optimistic in the face of uncertainty.
Section 11
Summary & Further Reading
UCB is a bandit policy: play the arm with the highest empirical mean plus a confidence bound. It balances exploration and exploitation without a tuning parameter and has strong regret bounds. Use it for A/B tests, recommendation, and adaptive allocation.
01
Paper
Original UCB analysis and regret bounds.
02
Book
Practical UCB and bandits for A/B testing and web.
03