Why You Can’t Always Trust Your Gut

Imagine you’re a contestant on the Let’s Make a Deal! gameshow, the old one hosted my Monty Hall. Monty offers you the chance to win a beautiful new car, but it’s behind one of three closed doors. Behind each of the other two doors is a goat. Let’s assume winning a new car is preferable to winning a goat.

You choose a door, and Monty (who, since he’s the host and in command, knows where the car is) opens one of the remaining two doors to reveal a goat. He offers you a choice: stay where you are, or switch to the other closed door. What do you do?

Basic instinct says there are two doors, so we have a 50-50 shot, so why switch, right? Go with your gut, your first intuition, right? Wrong. Always switch. Our odds double. Hard to believe, but your gut just lied to you.

In September 1991 a reader of Marilyn vos Savant’s column in the New York Times magazine Parade posed this very question. Marilyn answered correctly that the contestant should switch doors. Her answer provoked nearly 10,000 responses from readers, nearly all of them disagreeing with her, including a few academics lamenting both Marilyn’s and the nation’s lack of math skills, and who later had to retract their statements. (Note: if you want a full explanation, just Google “Monty Hall Problem.”)

What we generally refer to as intuition of “gut instinct” definitely doesn’t come from our gut. It comes from the soft-wired patterns in our brains. Our brains are powerful pattern makers, perpetually recognizing and constructing unconscious patterns, unbeknownst to us.

Intuition is a tapestry of unconscious patterns in action. Each of us holds a unique set of patterns that form our special point of view, our personal mindset built on our collective experiences in life.

Here’s how it works. Every new experience is automatically stored as data in our brain, to be grouped with other like data as it comes in. The groupings become patterns. Mental models. Paradigms. Experience is the input, patterns in the form of unconscious beliefs that govern our thoughts and behaviors are the output.

The good news is that intuition mostly works in a good way by helping us quickly arrive at the correct answer and take the best course of action. It’s especially useful for handling routine problems that don’t require deeper thought. (The last thing you need to worry about is a deep analysis of why you’re stuck in traffic–you just need a quick workaround.) Intuition is a huge part of our intelligence, and we’d be rather useless without it. It helps us rapidly sift and sort information into useful knowledge, according to whether it confirms or contradicts the strong patterns already embedded in our minds.

But therein lies the downside for more complex problem solving. For example, let’s say you’re playing a video game that gives you a choice: fight the alien superwarrior or fight three human soldiers in a row. The game informs you that your probability of defeating the alien superwarrior is one in seven. The probability of defeating a human soldier is one in two, a 50-50 shot. What do you do?

Most people would fight the human soldiers. It seems to make intuitive sense. The odds seem to be in your favor. But they’re not. Your probability of winning three battles in a row would be 1/2 cubed: 1/2 X 1/2 X 1/2, or 1/8 (one in eight). You have a better chance to win by fighting the alien superwarrior. It’s a simple problem of probability.

What leads us astray, gets in the way, and prevents us from solving the problem on the first pass is our reflexive thinking. We “know” the answer intuitively, at a glance. Generally speaking, as soon as we recognize a piece of information as being a part of a preexisting pattern, our mental models work to shortcut our thinking—-unconsciously jumping us ahead to a plausible conclusion. They then continue to work as a filter, screening in any information that supports our conclusion, screening out any information that conflicts with it or leads to another possibility.

By nature, the mind stays closed as long as possible!

The bottom line is that we tend to see only what we believe. And, if we limit our thinking to just what we believe, truly innovative thinking becomes a moonshot. Convention is convention because everyone believes it. Innovative thinking often runs counter to convention, counter to intuition.

Lesson: While it works fine for routine or benign problems, instinct must be tempered with logic and insight in order to solve tough problems.

In a followup column, I’ll show you how you can do just that in the context of business: predicting the success of a new offering.