How likely is it that two people would share a birthday in a party? If the party has 23 guests, the odds that two specific guests share a birthday party is 1 in 365. The odds that any two guests share a birthday is greater than 50%.
How likely is it that the same person wins the lottery twice? With the New Jersey lottery, the odds were in the region of one in 17 trillion. The odds that any person wins that lottery twice was 1 in 30.
How likely is it that you’d run into somebody you know during your vacation? If you expect to run into a specific friend or relative, the odds are low. But if you expect to meet anybody you know, the odds are higher than you think.
The odds of any coincidence happening is often much higher than the odds that a specific coincidence happens. Confusing the two is the human brain’s favourite past time.
Since it’s cricket world cup season, I recently came across a stat that left arm seamers were extraordinarily successful in world cups. Left arm seamers were the top wicket takers and scalped more than 20 wickets in 4 out of last 5 editions. Why is that so? This could easily lead people to interpret that something about the world cup’s atmosphere give left arm seamers an edge. However, the person who found that stat was presumably looking at a pool of data for any coincidence, and this one happened to turn up. When presented in isolation, this stat looks specific, and remarkable. But of all the patterns from cricket world cup bowling, at least one such finding has to emerge. That isn’t remarkable, it is inevitable.
Of course, these conclusions are relatively harmless in sports statistics (unless you’re the hapless right arm bowler who was dropped or you are betting big money on this “fact”). But in fields such as medicine, finance and recruitment, decisions made based on patterns in random data routinely cause us to make costly errors in judgement. The absence of a correlation may be used to rule out causation. But the presence of a correlation can never confirm causation.
The next time you run into somebody during your vacation, act surprised, but don’t be surprised. And on medical matters, get second opinions.
Inspiration: Fooled by Randomness – Nassim Taleb