The experimental mindset is the habit of treating beliefs as hypotheses and the world as testable. Instead of defending a view until it breaks, you state what would disprove it, run a check, and update. The stance is provisional: "this is my best guess given the evidence so far; here's what would change my mind." That shift — from conviction to testable claim — is what separates learning from confirmation. In complex systems, cause and effect are rarely obvious. The only reliable way to learn is to intervene, measure, and revise.
The mindset has three moves. First, make the hypothesis explicit. What do I believe? What would have to happen for that belief to be wrong? If you can't name a falsifying outcome, you have a belief, not a hypothesis. Second, design a test. The test should be capable of producing evidence that would change your view. Experiments that can only confirm are rituals. Third, update on the result. The hardest part. Most people run the test, get an answer, and then explain why the answer doesn't really apply. The experimental mindset demands that you change your mind when the evidence says so. The loop is: hypothesis → test → result → update. Then repeat.
The origin is the scientific method, but the application is broader. You don't need a lab or a p-value. You need the discipline to say "I could be wrong," to define wrong in advance, and to act when the data says you were. Organisations that lack this mindset optimise for being right in the room — winning the argument, protecting the plan. Organisations that have it optimise for being right in the world — their beliefs track reality because they keep testing. The experimental mindset is how you learn in systems where intuition and narrative are unreliable. Try something. See what happens. Change your mind when you have to.
Section 2
How to See It
You see the experimental mindset when people state falsification criteria before they have results. The tell is specificity: "If we don't see X within Y timeframe, we'll do Z." You also see it in the reaction to disconfirming evidence — do they update or rationalise? The absence of the mindset is equally visible: decisions made first, evidence marshalled after; experiments where every outcome is spun as success; and kill criteria that never get triggered no matter what the data shows.
Business
You're seeing The Experimental Mindset when a team ships a feature to a small segment, defines success and failure metrics in advance, and commits to rolling back if the failure condition is met. The discipline isn't the rollout — it's the pre-commitment. When the numbers come in weak, the team reverts instead of redefining success. The mindset shows up in the willingness to be wrong in a controlled way rather than right in a story.
Technology
You're seeing The Experimental Mindset when a platform runs hundreds of A/B tests and treats each as a hypothesis test. The infrastructure (random assignment, metrics, statistical guardrails) supports the mindset; the mindset is the culture that says "we don't know until we test." Teams that argue from first principles without running experiments lack it. Teams that run experiments but only celebrate wins and explain away losses lack it too. The mindset is in the update.
Investing
You're seeing The Experimental Mindset when an investor writes down the conditions under which they would exit or reduce a position before they enter. The thesis is a hypothesis; the market is the test. When the conditions are met, they act. The opposite is the investor who holds through deteriorating fundamentals because the story is still intact. The experimental mindset says: the story is a hypothesis. What would falsify it? Have we seen that? Then update.
Markets
You're seeing The Experimental Mindset when a policy or programme is piloted with clear success and failure criteria, and the pilot is actually used to decide scale-up or shutdown. Many "pilots" are performative — the decision to scale was made before the pilot. The experimental mindset insists the pilot can kill the idea. When the results are ambiguous or negative, the organisation stops or pivots instead of relabelling the pilot as "learning" and proceeding anyway.
Section 3
How to Use It
Decision filter
"Before committing to a course of action, ask: what would have to be true for this to work? What would show it's not working? Can I run a small test that could falsify it? If I can't answer, I'm not experimenting — I'm betting on a story. Define the test. Run it. Update on the result."
As a founder
Turn key assumptions into hypotheses and test them early. "Customers will pay for X." Run a pre-sale or a concierge test. "This feature will improve retention." A/B test it. "This channel will scale." Run a bounded experiment with a budget and a kill criterion. The mistake is building or scaling on untested beliefs. The second mistake is testing without pre-committing to what would make you stop. Write down the falsification criteria before you see the data. When the test says you're wrong, update. The experimental mindset compounds: each test improves your model of the world; founders who don't test repeat the same errors.
As an investor
Evaluate portfolio companies on whether they run real experiments or confirmation rituals. Do they define success and failure before the test? Do they act on negative results? When a founder says "we tested it," ask what would have counted as failure and whether that condition was ever triggered. The best companies treat strategy as a set of hypotheses and use small bets to test before big bets. The experimental mindset in a founder is a signal that they'll learn faster and waste less capital on wrong turns.
As a decision-maker
Reserve the right to be wrong. State your hypothesis and what would change your mind. When someone presents a plan, ask: what would cause us to abandon this? If the answer is "nothing" or vague, the plan is faith-based. Push for a test that could falsify the key assumption. When the test runs, honour the result. The hardest part of the experimental mindset is updating when the evidence goes against you. Build a culture where changing your mind on evidence is rewarded, not punished.
Common misapplication: Running experiments without falsification criteria. If every outcome is interpreted as support, you're not learning — you're collecting anecdotes. The mindset requires that some outcomes count as "we were wrong." Define them in advance.
Second misapplication: Confusing the experimental mindset with endless testing. The goal is to learn and then act. Some decisions are one-way doors and can't be tested cheaply; you make a bet and learn from the outcome. The mindset still applies: what would show the bet was wrong? When would we cut our losses? The experimental mindset is about how you update, not about avoiding all commitment.
Bezos made experimentation a core discipline at Amazon. The company runs thousands of experiments — on pricing, layout, recommendations, and operations — with random assignment and clear metrics. The mindset wasn't just "test things"; it was "treat every important assumption as a hypothesis and run a test that could disprove it." Bezos's 2016 letter to shareholders described the need to be wrong often: "If you're good at course-correcting, being wrong may be less costly than you think, whereas being slow is going to be expensive for sure." The experimental mindset at Amazon is the habit of running small, interpretable tests and updating strategy when the tests say the prior belief was wrong.
Hastings built Netflix's product and content strategy around rigorous experimentation. The company A/B tests recommendations, artwork, and features; the culture treats "we think" as a hypothesis until the test runs. Hastings has emphasised that data beats opinion — but only when the data comes from experiments that could have gone the other way. Netflix's willingness to kill shows or features that don't meet the bar is the experimental mindset in action: state the criterion, run the test, update. The result is a product that evolves on evidence rather than on the highest-paid person's opinion.
Section 6
Visual Explanation
The Experimental Mindset: Make the hypothesis explicit, define what would falsify it, run a test, and update on the result. The loop repeats; beliefs track reality when the update step is honest.
Section 7
Connected Models
The experimental mindset sits at the intersection of how we learn (scientific method, falsification), how we test (RCT, A/B testing, hypothesis), and how we respond to new information (feedback loops). The models below either formalise the loop, provide the tools for testing, or describe the update mechanism.
Reinforces
Scientific Method
The scientific method is the full cycle: observe, hypothesise, test, analyse, revise. The experimental mindset is the cultural and psychological adoption of that cycle — the habit of treating beliefs as hypotheses and the world as testable. You can know the method without having the mindset; the mindset is the default to use it. The two reinforce each other: the method gives structure, the mindset gives the willingness to be wrong and to update.
Reinforces
Principle of Falsification
Popper's criterion — a theory is scientific only if it could be disproven — is the intellectual backbone of the experimental mindset. If you can't state what would falsify your view, you're not holding a hypothesis; you're holding a belief. The mindset operationalises falsification: define the falsifying outcome in advance, run the test, and update when that outcome occurs. Falsification is the rule; the experimental mindset is the practice.
Leads-to
A/B Testing
A/B testing is the most common way to run experiments in product and growth: random assignment, treatment vs control, compare outcomes. The experimental mindset is what makes A/B tests useful. Without the mindset, teams run tests and then explain away bad results. With it, the test can actually change the plan. A/B testing is the tool; the experimental mindset is the discipline to let the tool change your mind.
Leads-to
Randomized Controlled Experiment
Section 8
One Key Quote
"Those among us who are unwilling to expose their ideas to the hazard of refutation do not take part in the game of science."
— Karl Popper, Conjectures and Refutations (1963)
The experimental mindset is the willingness to take that hazard. Exposing your ideas to refutation is exactly what the hypothesis–test–update loop does. The quote doesn't mention business or product — it's about science — but the same rule applies wherever learning matters. If you're unwilling to be wrong, you're not experimenting; you're performing. The mindset is the decision to play the game: state the idea, run the test, and let refutation do its job.
Section 9
Analyst's Take
Faster Than Normal — Editorial View
The experimental mindset is the main differentiator between teams that learn and teams that confirm. Everyone says they're "data-driven." The question is whether the data can change the plan. When success criteria are defined after the fact, when negative results are explained away, and when kill criteria never get triggered, the organisation is confirmation-driven. The experimental mindset is the discipline to define failure in advance and to act when it happens.
Pre-commitment is the lever. The power of the mindset comes from stating what would falsify your view before you see the result. Once the data is in, motivated reasoning takes over. "We didn't run long enough." "The segment was wrong." "External factors." The only defence is to write down the falsification criteria when you still have no stake in the outcome. Then when the result arrives, the criteria are already set. You either update or you're openly overriding your prior commitment.
Small tests compound. The mindset doesn't require big, formal experiments. It requires that you treat important assumptions as testable and run checks when you can. A pre-sale, a landing page, a two-week A/B test — each one updates your model of the world. Founders who do this repeatedly get better at distinguishing signal from noise. Founders who don't repeat the same strategic mistakes because they never ran the test that would have caught them.
The update step is the bottleneck. Most people can state a hypothesis. Many can design a test. Few actually change their mind when the test says they're wrong. The experimental mindset fails at the update. Organisations that reward consistency and punish reversals drive the mindset out. The culture that makes the mindset stick is one where "I was wrong" is a valid outcome and where changing your mind on evidence is treated as strength. Build that culture or the loop breaks.
Not everything can be tested. Some bets are one-way doors. The experimental mindset still applies: what would show the bet was wrong? When would we cut our losses? Defining the falsification condition forces clarity even when you can't run a clean experiment. The mindset is about how you hold beliefs and how you update — not about avoiding all commitment. Use it to make better bets and to exit bad ones faster.
Section 10
Summary
The experimental mindset is treating beliefs as hypotheses and the world as testable. Make the hypothesis explicit and state what would falsify it. Design a test that could produce that evidence. Run it. Update on the result. The loop — hypothesis → test → result → update — only works if you honour the update. Pre-commit to falsification criteria before you have data. When the test says you're wrong, change your mind. The mindset compounds: each cycle improves your model of reality. Without it, you optimise for being right in the room. With it, you optimise for being right in the world.
Popper's formulation of falsification as the criterion for scientific claims. The experimental mindset is the application of that criterion to belief and action: state what would disprove you; update when it happens.
Kahneman's treatment of how we avoid updating — confirmation bias, narrative fallacies, overconfidence. The book explains why the experimental mindset doesn't come naturally and why the update step is the one that fails.
Ries popularised build–measure–learn and the idea of testing assumptions before scaling. The experimental mindset in startup form: treat the business model as a set of hypotheses, run experiments, and pivot or persevere on evidence.
Tetlock's research on who updates well. Superforecasters hold beliefs provisionally, state probabilities, and revise when new evidence arrives. The experimental mindset applied to prediction.
Harford argues that success in complex environments requires trial and error — trying things, learning from failure, and adapting. The experimental mindset at the level of strategy: try, fail, learn, try again.
RCTs are the gold standard for causal inference: randomise, intervene, measure, compare. The experimental mindset extends the same logic beyond formal RCTs. When you can't randomise, you still state the hypothesis and what would falsify it; you still update on evidence. The RCT is the ideal; the mindset is the stance that you should approximate it when possible and be clear about limits when not.
Reinforces
Hypothesis
A hypothesis is a testable claim — something that could be right or wrong and that you can check. The experimental mindset is the habit of framing beliefs as hypotheses: explicit, falsifiable, and subject to revision. Without the hypothesis frame, "we believe X" is a closed position. With it, "we hypothesise X; here's how we'll test it" opens the loop. The model gives the form; the mindset gives the default to use it.
Tension
[Feedback](/mental-models/feedback) Loops
Feedback loops describe how output feeds back to influence the next input. The experimental mindset is a feedback loop for beliefs: result → update → new hypothesis → test. The tension: feedback loops can be reinforcing (confirming) or balancing (correcting). The experimental mindset aims for the balancing loop — evidence corrects belief. The risk is turning the loop into confirmation: only notice evidence that supports the prior. The mindset requires designing the loop so that disconfirming evidence actually flows back into the update.
