Base rate neglect and career regret

Read the description of Shyam – an Indian person selected at random. Shyam’s neighbour describes him as follows:

Shyam is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.

Is Shyam more likely to be a librarian or a farmer?

Most people would guess Shyam to be a librarian – his personality matches that of a stereotypical librarian. But most people also end up neglecting statistical information that is highly relevant here. The number of farmers in India is at least 30 times more than the number of librarians. Given Shyam was an Indian person selected at random, it is far more likely for him to have been a farmer than a librarian. Although this is objective truth, it is hard for our minds to digest.

Daniel Kahneman provides the example above (which I have adapted for an Indian setting), to illustrate base rate neglect. The faster, more automatic parts of our brains jump to conclusions all the time without pausing to think about statistical information. Experts across fields, including statisticians, are prone to committing this error.

Neglecting base rate information while choosing careers, however, can be a deadly mistake. At any point, careers that are most accessible are also the ones that are statistically significant. In most parts of India, people are likely to have easier access to a career in farming than within libraries. And yet, when our minds and most people around us cannot reconcile our personalities with the jobs we are in, we end up feeling that we are stuck in the wrong job.

A good reason why “follow your passion” is bad career advice is because of base rate neglect. The alternative here is that every student ought to be given information about base rates and where they are headed in the next five years before selecting their degrees of specialization. Most governments can collect this information and make them available quite easily. This information could also inform policy in education.

But most importantly, it can rescue large sections of people feeling as though they are stuck in the “wrong jobs” for large portions of their lives. The first requirement for any dream job is that the market needs it.

Inspiration: Thinking, Fast and Slow – Daniel Kahneman

3 thoughts on “Base rate neglect and career regret

  1. The kind of system you are suggesting would encourage “fitting into the mould” mentality. How would you have entrepreneurship in such a system?


  2. Actually this model sounds interesting.. Today a student simply does not know where to focus and learn to increase his employability. While there are few basic skills that a child should learn to suceed- empathy, team work, listening skills and the ability to unlearn and re-learn, it would be helpful if they are provided with the data on the market demand for jobs. However with the job and technology landscape changing at a rapid pace, at least this generation would have 2-3 different jobs if they have to thrive meaningfully.

    Liked by 1 person

    1. Yes – it would be a challenge for sure in current times. About 30 years back, with nations engaged in 5 year planning, this would have been easier to execute.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s