Monday, June 05, 2017

Survivorship Bias - The Pitfalls Of Learning From The Lucky Winners

A Tweet about survivor bias led me to an old post on "You Are Not So Smart."  It's just good reading with lots of information.

It's a long, somewhat rambling, post.  But it's well told and brings lots of ideas together.  It's about how survivor bias leads us, often, to faulty conclusions.  The reason?  Because we only hear from the survivors and not those who failed, which often can be the vast majority. Starts with a long story about:
"The official name for the people inside the apartment was the Statistical Research Group, a cabal of geniuses assembled at the request of the White House and made up of people who would go on to compete for and win Nobel Prizes. The SRG was an extension of Columbia University, and they dealt mainly with statistical analysis. The Philadelphia Computing Section, another group made up entirely of women mathematicians, worked six days a week at the University of Pennsylvania on ballistics tables. Other groups with different specialties were tied to Harvard, Princeton, Brown and others, 11 in all, each a leaf at the end of a new branch of the government created to help defeat the Axis – the Department of War Math."
One of their more tangible problems was how to better protect WWII fighter planes in the Pacific.  The problem was the military saw the bullet holes only on the planes that survived and tried to protect the planes better in those places.  Mathematicians pointed out that those planes had made it back, so those hits were survivable.  They needed reinforcement in the places the surviving planes hadn't been hit.

It then goes on to give lots and lots of other examples of survivor bias.

  • in employer surveys (people who quit don't get surveyed)
  • old art and literature is better than modern stuff (bad stuff in the past disappeared and only the 'not bad' survived, while the current junk is still all around)
  • 'how to succeed books' from the successful (you miss the stories of the many ways to fail, and success seems much easier than it really is)
  • lottery stories

There's also an interesting section on luck and lucky people.
"Wiseman speculated that what we call luck is actually a pattern of behaviors that coincide with a style of understanding and interacting with the events and people you encounter throughout life. Unlucky people are narrowly focused, he observed. They crave security and tend to be more anxious, and instead of wading into the sea of random chance open to what may come, they remain fixated on controlling the situation, on seeking a specific goal. As a result, they miss out on the thousands of opportunities that may float by. Lucky people tend to constantly change routines and seek out new experiences. Wiseman saw that the people who considered themselves lucky, and who then did actually demonstrate luck was on their side over the course of a decade, tended to place themselves into situations where anything could happen more often and thus exposed themselves to more random chance than did unlucky people. The lucky try more things, and fail more often, but when they fail they shrug it off and try something else. Occasionally, things work out."
I'm not totally sure about this.  I'd guess there are lots of 'unlucky' folks who take crazy risks that get them into serious trouble.  But maybe Wiseman can explain how that still fits his model.

There's history in this post, plus some good science lessons.  Basically this seems like a specific kind of generalization based on a biased sample.

There was something familiar about the site and so I did check and found I had written about the book You are Not So Smart a few years ago.


  1. I completely believe in the psychology of luck. It isn't that lucky people take crazy risks, it's that they understand risk better and are more comfortable with it. They appraise it more accurately and thus it works out well for them more often than unlucky people who don't understand it and do, indeed, take crazy, uninformed, poorly considered risks.

    Moreover, lucky people seize opportunities (not necessarily risky ones) that others don't even see, or are too afraid to act on.

    I believe it because it is the story of my life. I've done all sorts of amazing things and had all sorts of amazing things fall in my lap. I'm incredibly lucky, happily married for 35 years, traveled the world, and a self-made millionaire. But I fit Wiseman's profile to a 'T'. I've made my own luck.

    1. Anon, thanks for dropping by and the link to the article. I agree that lucky people are really those who are ready to take advantage when opportunity knocks and they get out there where the opportunities are.
      But as a white male I'm also very aware that I face a lot fewer obstacles than women and people of color. I'm also aware that luck I have nothing to do with - good genes, parents who encouraged me to explore and provided me with lots of experiences of different things - also helped shaped me into a person who is better able to than many to see and then take advantage of opportunities that come my way. Lots of kids learn not to take any risks or step out of the routine because their parents punish them if they do.
      So I'm thankful I was lucky to get good parents and to arrive in a culture that was designed for people like me and I recognize that being 'lucky' isn't as easy for others.


Comments will be reviewed, not for content (except ads), but for style. Comments with personal insults, rambling tirades, and significant repetition will be deleted. Ads disguised as comments, unless closely related to the post and of value to readers (my call) will be deleted. Click here to learn to put links in your comment.