Survivor bias
In his lecture titled “The Soul of statistics” Joseph Blitzstein talks about a survivor bias (or conditioning more broadly) Dr. Derek Muller also talks about various examples of Korean houses in Bukchon Hanok Village on his YouTube
Survivor bias is today’s topic and the following was written mostly by GPT 4.
Survivor Bias: What Remains Tells Only Half the Story
Imagine walking through a forest and noticing the tallest trees. You marvel at their height and strength, thinking that this is the natural order of things. But what about the saplings and smaller trees that didn’t survive? This is the essence of survivor bias.
What is Survivor Bias?
Survivor bias, or survivorship bias, is a logical error of focusing on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility. This can lead to false conclusions in numerous different ways.
A Classic Example: WWII Airplanes
During World War II, military officials examined planes returning from combat missions to determine where they were most frequently hit by enemy fire. The plan was simple: reinforce these areas to improve the aircraft’s survival rate. The bullet holes were predominantly in the wings, body, and tail. So, it might seem logical to reinforce these parts.
However, a statistician named Abraham Wald pointed out a flaw in this reasoning. The planes they were inspecting had survived. The real question was: where were the bullet holes on the planes that didn’t return? Wald hypothesized that the missing airplanes had been hit in the engine, a critical area absent of damage in the returning planes. By only looking at the survivors, the military had almost made a grave error.
Why Does It Matter?
Survivor bias can skew our understanding and lead to incorrect conclusions in various fields:
Business: When studying successful companies, we might conclude that their practices are best. But what about companies that followed the same practices and failed?
Medicine: If we only focus on patients who return for follow-up after treatment, we might miss side effects or outcomes in those who didn’t return.
Culture: Celebrating only the top artists or authors might make us think that a particular style or theme is the key to success, overlooking other potential talents.
Overcoming Survivor Bias
Awareness is the first step. Whenever you’re examining successes, ask yourself: “What am I not seeing?” Seek out the failures, the unseen, the unreturned. By considering the whole picture, not just the apparent survivors, you get a clearer, more accurate view of reality.
In conclusion, while it’s natural to focus on winners and success stories, it’s crucial to remember the unseen and unspoken failures. They often hold the most valuable lessons.