Skip to content
Faster Than Normal
General Thinking & Meta-Models

Reproducibility

Model #0737Category: General Thinking & Meta-ModelsDepth to apply:

By Updated 3 sources

4 min read

Contents

  1. 1. Core Idea
  2. 2. How to See It
  3. 3. How to Use It
  4. 4. Founders & Leaders
  5. 5. Connected Models
  6. 6. One Key Quote
  7. 7. Summary & Further Reading
General Thinking & Meta-Models
Section 1

Core Idea

Reproducibility is the requirement that a result can be independently obtained using the same methods. If you run the experiment again — different lab, different team — and get the same answer, the finding is real. If you can't, it might be noise, error, or fraud. This is the bedrock of the scientific method, but founders should apply it with equal rigour: can your growth result be reproduced next quarter, or was it a one-off? Can your hiring process reliably identify good candidates, or did you get lucky? Reproducibility separates systems from accidents. Any claim — scientific, strategic, operational — that cannot survive independent replication should be held at arm's length until it can.

Get Faster Than Normal by email

Ideas from founders and companies.

Free newsletter. Unsubscribe anytime.

Or open the full subscribe page.

Section 2

How to See It

Growth
You're seeing it when a marketing channel delivers one great month but can't repeat it. The result was real; the system wasn't — which means you can't compound it.
Hiring
You're seeing it when a hiring process that produced one great team member fails to produce another. The first hire was luck; reproducibility requires a reliable process.
Product
You're seeing it when a feature launch drove engagement once but the next three launches didn't move the needle. Without reproducibility, you're pattern-matching on noise.
Section 3

How to Use It

Before scaling any result, test whether it reproduces. Run the experiment again with a different cohort, market, or team. If the result holds, invest. If it doesn't, you learned cheaply. Build reproducibility checks into your operating rhythm: quarterly retros that ask not "what worked?" but "what worked and would work again?" This filters signal from luck.
Decision filter
"Can this result be independently reproduced — or are we scaling an accident?"
As a founder
Before pouring resources into a result, reproduce it. Test the winning campaign in a new market. Run the hiring process again. If the outcome holds, you have a system worth scaling. If it doesn't, you caught the error before it compounded.
Section 5

Founders & Leaders

Thomas EdisonInventor; founder, General Electric; holder of 1,093 patents
Edison's laboratory method was built on reproducibility before the term existed in its modern sense. He didn't trust a single successful experiment — he required results to be replicated across conditions before committing to commercialisation. The famous "ten thousand ways that don't work" narrative obscures the real discipline: Edison tested, recorded, and demanded that results hold up under repetition before he invested in manufacturing. For founders, this is the gap between a promising pilot and a scalable system. One successful sales call, one viral post, one productive hire — none of these mean anything until they reproduce. Edison's contribution wasn't persistence alone; it was the insistence that any result worth scaling must first survive replication. Build that standard into your operating culture and you filter luck from systems early.
Section 7

Connected Models

Reinforces
Replication Crisis
The replication crisis is what happens when reproducibility standards slip across an entire field. It demonstrates the systemic cost of accepting unreplicated results.
Reinforces
Scientific Method
The scientific method requires reproducibility as its validation layer. A hypothesis survives not by being proposed but by being independently confirmed.
Tension
P-hacking
P-hacking produces results that pass significance tests but fail reproduction. It is the primary mechanism by which irreproducible findings enter the record.
Section 8

One Key Quote

"Extraordinary claims require extraordinary evidence — and evidence means it can be checked."
— Carl Sagan
Section 11

Summary & Further Reading

Reproducibility requires that a result survives independent replication. Before scaling any finding — growth result, hiring process, product insight — test whether it reproduces. This separates systems from accidents.
01
Science Fictions — Stuart Ritchie (2020)
Book
How the failure of reproducibility undermines modern science.
02
Edison: A Biography — Matthew Josephson (1959)
Book
Edison's laboratory method and insistence on repeatable results.
03
The Demon-Haunted World — Carl Sagan (1995)
Book
On the tools of sceptical thinking, including reproducibility as a safeguard.

Why this matters next

mental modelsScientific Method

Reproducibility applied the Scientific Method mental model

mental modelsNarrative

Reproducibility applied the Narrative mental model

mental modelsReproducibility

Reproducibility applied the Reproducibility mental model

mental modelsCost

Reproducibility applied the Cost mental model

mental modelsP-hacking

Reproducibility applied the P-hacking mental model

mental modelsSystematic Review

Reproducibility applied the Systematic Review mental model

Frequently asked questions

What is Reproducibility?

Reproducibility is a mental model used for better thinking and decision-making.

How do you apply Reproducibility?

To apply Reproducibility, identify situations where this framework is relevant, then use it as a lens to evaluate your options and decisions. The model is most useful when combined with other complementary mental models.

What category does Reproducibility fall under?

Reproducibility falls under the General Thinking & Meta-Models category of mental models. Other models in this category can be found on the General Thinking & Meta-Models hub page.

Why is Reproducibility important?

Reproducibility is important because it provides a structured way to think about problems that would otherwise be approached with intuition alone. Understanding this model helps you avoid common reasoning errors and make better decisions.

Continue exploring

5W

Mental model

5 Whys

A root-cause analysis technique that drills past symptoms by asking why five suc

AW

Mental model

All Models Are Wrong

George Box s insight that every model is a simplification of reality — the quest

AL

Mental model

Amara's Law

We tend to overestimate the effect of a technology in the short run and underest

EM

Mental model

Eisenhower Decision Matrix

A prioritization framework that sorts tasks into four quadrants based on urgency

FT

Mental model

First Principles Thinking

Breaking down complex problems to their most fundamental truths, then reasoning

GL

Mental model

Goodhart's Law

When a measure becomes a target, it ceases to be a good measure — incentives shi

Get Faster Than Normal by email

Ideas from founders and companies.

Free newsletter. Unsubscribe anytime.

Or open the full subscribe page.

Popular Mental Models

First Principles ThinkingOccam's RazorCircle of CompetenceInversionConfirmation BiasSecond-Order ThinkingDunning-Kruger EffectSurvivorship BiasPareto PrincipleOpportunity Cost