Queuing Theory

Queuing theory is the study of waiting lines: how jobs, requests, or customers arrive, how they are served, and how long they wait. The core result is that wait times and queue length explode when utilisation approaches 100%. A server that is 80% utilised may have manageable queues; at 95% utilisation, small spikes in arrival rate cause long delays. The relationship is nonlinear: doubling utilisation does not double wait time; it can increase it by an order of magnitude. When building and scaling systems — whether call centres, servers, factories, or support queues — the implication is that running at full capacity is unstable. You need slack (idle capacity) to absorb variance.

The basic setup: arrivals follow some process (e.g. Poisson), service takes some distribution of time, and there are one or more servers. Key metrics include utilisation (ρ = arrival rate / service rate), average wait time, and queue length. Little's Law links them: average number in system = arrival rate × average time in system. As ρ → 1, wait time and queue length go to infinity unless there is variability in arrivals or service. In practice, the lesson is to design for utilisation well below 100% (e.g. 70–85%) or to add capacity (more servers, faster service) so that the system can absorb bursts without collapsing.

Use queuing theory when you are designing or scaling a system where demand is variable and service takes time. Identify the bottleneck (the queue or the server), measure arrival and service rates, and size capacity so that utilisation stays in a range where wait times remain acceptable. Building and scaling without this lens often leads to systems that work in the average case and fail under load.