Upper Confidence Bound

UCB (Upper Confidence Bound) is a simple policy for the multi-armed bandit: for each arm, compute an index = estimated mean + confidence bound (e.g. proportional to √(log n / n_i), where n is total pulls, n_i pulls of arm i). Play the arm with the highest index. The bound encodes uncertainty: underplayed arms get a higher bonus, so they get tried; as n_i grows, the bound shrinks and the policy exploits the best arm. No tuning of exploration rate—the bound is derived from concentration inequalities. Near-optimal regret in many settings and a workhorse for A/B tests, recommendation, and adaptive allocation. Sub-entry under explore–exploit.

Maintain a running mean and count per arm; at each step compute UCB = mean + c√(ln N / n_i) (c a constant, N total pulls, n_i pulls of arm i). Play the arm with max UCB. Works for bandits, A/B/n tests, and any setting where you learn by trying. Prefer UCB when you want a simple, parameter-light policy with good theoretical guarantees.

UCB is a bandit policy: play the arm with the highest empirical mean plus a confidence bound. It balances exploration and exploitation without a tuning parameter and has strong regret bounds. Use it for A/B tests, recommendation, and adaptive allocation.