By: Ryan Gavin, CFA
Portfolio Manager
Imagine a disease that affects 1 out of every 1,000 people. A test for it is 99% accurate. You take the test and it comes back positive. Do you think you probably have the disease?
Most people say “yes.” In reality, the probability is only about 9%.
To see why, we have to look at the base rate of the population. The test correctly identifies the one sick person (~1 true positive). Of the 999 healthy people, even with 99% accuracy, the test will incorrectly flag 1% of them (~10 false positives). Out of 11 people with a positive result, only one is actually sick. This is definitely not medical advice.
The mistake is ignoring how rare the disease was to begin with. We get so blinded by the 99% accuracy of the signal that we forget to account for the 0.1% frequency of the event.
Librarian or farmer?
Daniel Kahneman’s excellent book Thinking Fast and Slow provides a classic example.
“Imagine a man named Steve. He is very shy, withdrawn, helpful, but has little interest in people or the world of reality. He is a meek and tidy soul with a need for order and a passion for detail.”
Is Steve more likely to be a librarian or a farmer? Most people say “librarian” because Steve sounds like a librarian, but in the real world, the number of male farmers greatly exceeds the number of male librarians. Even though Steve projects the “vibe” of a librarian, the sheer number of farmers means it is statistically more likely that he is a farmer who just happens to be shy.
False alarm or true warning?
The same problem appears whenever we try to detect a rare signal in a noisy system. Financial markets are full of situations like this.
An economist may predict a recession in the near future, but we should first consider how often recessions occur. In the United States, economic expansions are far more frequent than recessions. That means even a historically reliable recession indicator – if such a thing exists –can generate more false alarms than true warnings. When the event itself is relatively rare, a strong signal does not automatically imply a high probability outcome.
The SPIVA Scorecard and Morningstar’s Active/Passive Barometer show that most active managers underperform their benchmarks after fees. When we see a fund that has outperformed, it may reflect genuine skill, but if persistent alpha is uncommon, many instances of outperformance will simply be the result of randomness. If hundreds or thousands of managers are trying to beat the market, random variation alone can produce a handful of managers with impressive track records over a given period. Without considering the base rate of true skill, it is easy to mistake those lucky streaks for evidence of forecasting ability.
How common?
Focusing on base rates forces us to start with the broader statistical reality before reacting to a compelling signal. Whether evaluating a recession forecast, an outperforming fund manager, or any other uncertain outcome, the first question should be how common the event is to begin with. Ignoring that context can lead us to mistake noise for insight. Incorporating base rates does not eliminate uncertainty, but it helps anchor our decisions in probability rather than intuition.
