A test or decision has four outcomes: true positive (correctly say "yes" when the state is yes), true negative (correctly say "no" when the state is no), false positive (say "yes" when the state is no — Type I error), and false negative (say "no" when the state is yes — Type II error). No real-world test is perfect; you trade off the two errors. Lowering the bar for "yes" increases true positives and false positives; raising it increases true negatives and false negatives. The cost of each error depends on context. In screening, a false positive may mean unnecessary follow-up; a false negative may mean a missed case. In hiring, a false positive is a bad hire; a false negative is a missed good candidate. Strategy is choosing which error to minimise when you cannot eliminate both.
Base rates matter. When the thing you are testing for is rare, even a good test will produce many false positives among positive results (positive predictive value falls). When it is common, false negatives can dominate. So the optimal threshold and the meaning of "positive" depend on prevalence and on the cost of each error. One threshold does not fit all contexts.
Practical use: make the trade-off explicit. What is the cost of a false positive vs a false negative in this decision? Set thresholds and process accordingly. When interpreting "positive" results, combine test performance with base rate (Bayes). Do not treat sensitivity and specificity as enough without prevalence and costs.
Section 2
How to See It
False positives and false negatives show up whenever a binary decision (yes/no, accept/reject, signal/noise) is made with imperfect information. Look for threshold choices and for settings where one kind of error is systematically underweighted.
Business
You're seeing False Positives & False Negatives when a sales team loosens qualification to fill the pipeline and ends up with more leads but lower conversion — more false positives (leads that never buy). Or when they tighten qualification and miss good customers — more false negatives. The threshold choice is explicit; the trade-off is in the conversion and opportunity cost.
Technology
You're seeing False Positives & False Negatives when a fraud system flags transactions. Tune it to catch more fraud and you get more false positives (blocked good customers). Tune it to reduce friction and you get more false negatives (missed fraud). The team is trading off two error types; the right balance depends on the cost of each.
Investing
You're seeing False Positives & False Negatives when an investor screens deals. Say yes to more and you back more bad companies (false positives); say no to more and you pass on more good ones (false negatives). The bar is a threshold; moving it shifts the mix of errors. In a world where good deals are rare, most "yes" decisions will be false positives unless the bar is very high.
Markets
You're seeing False Positives & False Negatives when a regulator approves or blocks products. Approve too easily and bad products reach the market (false positives); block too aggressively and good products are delayed or killed (false negatives). Policy is the choice of which error to prioritise and at what cost.
Section 3
How to Use It
Decision filter
"For any binary decision with imperfect information, define the cost of a false positive and the cost of a false negative. Set the threshold (or process) to minimise total expected cost given base rates. When you see 'positive' results, combine test accuracy with prevalence — do not ignore base rate."
As a founder
Design funnels and filters with the trade-off in mind. Lead qualification, hiring, and feature flags all have thresholds. If false positives are cheap (e.g. extra leads to disqualify later), lower the bar; if they are expensive (bad hire, wrong launch), raise it. If false negatives are expensive (missing a key hire, missing fraud), accept more false positives to reduce them. State the costs and base rates explicitly so the threshold is a choice, not a default.
As an investor
Every "yes" is a bet that this is a true positive. In a world where great companies are rare, most "yes" will be false positives unless the bar is very high. The question is the cost of a false positive (capital and time lost) vs the cost of a false negative (missing a winner). Set the bar and process to reflect that. When you see pattern-matching or scoring, ask: what is the implied base rate and what happens to positive predictive value when base rate is low?
As a decision-maker
Institutionalise "cost of each error" in gate decisions: hiring, go/no-go, launch, partnership. Make the threshold explicit and revisit it when base rates or costs change. When a test or screen has high sensitivity but the condition is rare, expect many false positives among positives — use Bayes to interpret results.
Common misapplication: Optimising only for one error type. Reducing false positives often increases false negatives, and vice versa. The goal is to minimise expected cost (or maximise expected value) given both errors and their costs.
Second misapplication: Ignoring base rate when interpreting positives. When the thing you are testing for is rare, most positive results can be false positives even with a good test. Always combine test performance (sensitivity, specificity) with prevalence.
Thorp used probability and decision theory to design betting and investing strategies. Understanding false positives and false negatives — and the role of base rates — was central. In blackjack and in markets, the edge is small and the variance is high; mistaking noise for signal (false positive) or missing real signal (false negative) had direct financial cost. He tuned thresholds and position sizing to the actual probabilities and costs.
Buffett's "wait for the right pitch" is a high bar that accepts many false negatives (passed opportunities) to avoid false positives (bad investments). He explicitly prefers missing some good deals to buying bad ones. The threshold is set by the asymmetric cost of the two errors.
Section 6
Visual Explanation
A 2×2 table: rows = true state (positive / negative), columns = decision (positive / negative). Top-left: true positive; top-right: false negative; bottom-left: false positive; bottom-right: true negative. Sensitivity = TP / (TP + FN); specificity = TN / (TN + FP). Moving the decision threshold slides the split between the two error cells. The curve of sensitivity vs (1 − specificity) is the ROC curve; the optimal point is where expected cost is minimised given base rate and error costs.
Section 7
Connected Models
False positives and false negatives connect to probability, evidence, and decision cost. These models reinforce the trade-off or extend how to set thresholds and interpret results.
Reinforces
Bayes Theorem
Bayes combines prior (base rate) with likelihood (test performance) to get posterior (probability of true state given result). Positive predictive value is the posterior for "positive" results; it falls when base rate is low. False positive/negative analysis is Bayesian decision-making with two error types.
Reinforces
Base Rate
Base rate is the prior probability of the state before the test. It determines how many of your "positives" are true vs false positives. Ignoring base rate leads to over- or under-reaction to test results. False positive/negative reasoning requires base rate to set thresholds and interpret outcomes.
Tension
Signal vs Noise
Signal vs noise is the problem of distinguishing real signal from random fluctuation. False positives are noise misclassified as signal; false negatives are signal misclassified as noise. The tension: lowering the bar for "signal" increases both hits and false alarms; raising it reduces false alarms but misses signal.
Tension
Confirmation Bias
Confirmation bias is the tendency to accept evidence that supports a prior view. That can increase false positives (we accept weak evidence as confirmation) or false negatives (we reject disconfirming evidence). The tension: the same bias distorts how we set thresholds and interpret results.
Sagan's dictum is a threshold rule: when the base rate of a claim is very low (e.g. "this is a true positive"), you need strong evidence before accepting it. Otherwise most acceptances will be false positives. The same logic applies to any rare-but-important signal: set the bar high enough that positive predictive value is acceptable.
Section 9
Analyst's Take
Faster Than Normal — Editorial View
Make the trade-off explicit. Most teams use an implicit threshold (e.g. "we only hire people who pass the bar") without stating the cost of a false positive (bad hire) vs a false negative (missed great candidate). State both costs and set the bar. Revisit when the market or role changes.
In venture, most "yes" are false positives. Great companies are rare. So even a skilled investor will back more losers than winners. The bar should be high enough that the cost of false positives (capital, time) is acceptable given the payoff from true positives. Do not confuse "we said yes" with "this is a true positive."
Base rate dominates when the signal is rare. When you are screening for something rare (fraud, disease, exceptional deal), most positives will be false positives unless the test is very specific. Use Bayes: combine test performance with prevalence. Communicate this when presenting "positive" results to avoid overreaction.
Asymmetric costs justify asymmetric thresholds. If a false negative is 10× worse than a false positive (e.g. missing a security breach), the threshold should favour catching more positives and accepting more false alarms. If the reverse, favour fewer positives. The threshold is not "balanced" by default; it should reflect the cost ratio.
Section 10
Test Yourself
Is this mental model at work here?
Scenario 1
A hiring manager raises the bar for interviews after two bad hires. Six months later, the team has fewer new hires and complains about slow recruiting. The manager says the bar was necessary to avoid more bad hires.
Scenario 2
A rare disease affects 1 in 10,000. A test is 99% sensitive and 99% specific. A patient tests positive. The doctor says they are almost certainly sick.
Section 11
Summary & Further Reading
False positives (say yes when no) and false negatives (say no when yes) are the two error types in binary decisions. You trade them off by setting a threshold; the optimal threshold depends on the cost of each error and on base rate. When the condition is rare, combine test performance with prevalence so that "positive" is interpreted correctly. Make the trade-off and base rate explicit in strategy and process.
Formal treatment of decision under uncertainty, loss functions, and the trade-off between Type I and Type II errors. Foundational for threshold and decision design.
Duke on decision-making under uncertainty and how to separate outcome from decision quality. Relevant to interpreting "positive" results and setting decision thresholds.
Leads-to
Statistical Significance
Statistical significance is a threshold for rejecting the null hypothesis — a choice that controls Type I error (false positive) at a chosen level. Power (1 − Type II error) is then determined by sample size and effect. Significance and power are the formalisation of the false positive/false negative trade-off in inference.
Leads-to
Cost of Capital
In investment, the cost of a false positive is capital deployed in a bad project; the cost of a false negative is foregone return from a good project. Cost of capital and opportunity cost define the two error costs. Setting the hurdle rate and deal bar is setting the threshold in a false positive/false negative framework.
