Computer Science & Algorithms
Section 1
Core Idea
Optimization algorithms search for the best solution in a space of possibilities: maximise reward or minimise cost. Greedy methods (e.g. hill climbing) move toward better neighbours and stop at a local optimum — fast but can get stuck. Random restart runs hill climbing from many starts to improve the chance of finding a global optimum. Simulated annealing allows occasional downhill moves early on, then "cools" to reduce randomness and settle near a good solution. Metropolis and related MCMC methods use probabilistic acceptance to explore the space. The tradeoff: exploitation (refining the current best) vs exploration (trying new regions). Different algorithms tune that tradeoff differently.
Free playbook
Get The Business Model Playbook
58 business models, one visual page each: how the money flows, the metrics that matter, and who runs it. Free when you join the Faster Than Normal email.
Free. No spam. Unsubscribe anytime.
Section 2
How to See It
Deciding & Judging
You're seeing Optimization Algorithms when a process improves a metric step by step (hill climbing), or when a team deliberately tries multiple starting points or allows temporary setbacks to escape local optima (restart, annealing).
Understanding & Analyzing
You're seeing Optimization Algorithms when ML training or parameter tuning uses gradient descent, random restarts, or temperature schedules — the same ideas as hill climbing and simulated annealing in abstract space.
Section 3
How to Use It
When the landscape has many local optima, pure hill climbing can trap you. Use random restart (many independent runs) or simulated annealing (allow bad moves early, then reduce randomness). When the objective is noisy or expensive to evaluate, balance exploitation and exploration explicitly. Match the algorithm to the problem: smooth single peak → hill climbing; rugged landscape → restart or annealing.
Decision filter
"Are we stuck at a solution that's good locally but might not be global? If yes, we're in local-optimum territory — try restarts or annealing (explore more early, then refine). If the space is simple, greedy may be enough."
As a founder
Strategy and product search are optimization: you're climbing a landscape of outcomes. If you're stuck, you may be at a local max. Try restarts (new segments, new positioning) or annealing (experiment with "worse" short-term moves to test new regions). Avoid over-exploiting one hill before exploring the terrain.
Section 5
Founders & Leaders
Jim SimonsFounder, Renaissance Technologies; mathematician
Simons's edge came from optimization at scale: search over strategies, parameters, and data with rigorous use of statistics and algorithms. Renaissance uses the same conceptual toolkit — local search, exploration vs exploitation, and handling noise — in quantitative markets. Founders can adopt the mindset: treat strategy as an optimization over a rugged landscape; use restarts and structured exploration to avoid getting stuck at local optima.
Section 7
Connected Models
Reinforces
Global & Local Maxima
Optimization algorithms are the machinery for finding maxima. Hill climbing finds local maxima; random restart and simulated annealing improve the chance of reaching a global one. The model is the landscape; the algorithms are how we search it.
Tension
Explore-exploit Tradeoff
Hill climbing is pure exploit; annealing and restarts add exploration. The tension: too much exploit → stuck locally; too much explore → never converge. Algorithms formalise how to balance the two.
Leads-to
Theory of Constraints
TOC finds the bottleneck and improves there — a form of local optimization. Optimization algorithms generalise: when improvement at the bottleneck stalls, consider it a local optimum and "restart" or "anneal" (e.g. relax a different constraint) to explore the system.
Section 8
One Key Quote
— Kirkpatrick et al., Simulated Annealing (1983)"Simulated annealing is a general approach to combinatorial optimization that allows occasional acceptance of worse solutions in order to escape local optima." The cooling schedule tunes exploration over time.
Section 11
Summary & Further Reading
Optimization algorithms search for best solutions: hill climbing (greedy), random restart (many runs), simulated annealing (controlled exploration then refinement). Use them when the landscape has local optima; match the algorithm to the ruggedness of the problem.
01
Book
Standard reference for greedy algorithms, local search, and complexity of optimization.
02
Paper
Foundational paper on simulated annealing and escaping local optima.
03
Book
Applies optimization algorithms (including stopping and exploration) to life and business decisions.