Dr. Tomas Chamorro-Premuzic, an international authority in personality profiling and psychometric testing, believes all entrepreneurial activity starts at an individual level and that some individuals have much more potential for it than others. He’s the co-founder of META Profiling (META stands for “Measure of Entrepreneurial Talent and Abilities”). The company uses an online profiling tool to help identify entrepreneurial potential to help businesses nurture and retain their entrepreneurial talent.

When Chamorro-Premuzic’s new book Confidence: Overcoming Low Self-Esteem, Insecurity, and Self-Doubt crossed my desk, I assumed it was going to tell me what the title promised. I wasn’t prepared for what it really is: an expose of the dark side of confidence. I absolutely loved it, because it shatters so much incorrect but conventional wisdom with key scientific research.

I recently caught up with Chamorro-Premuzic to speak with him about confidence, success and entrepreneurial talent.

What’s the dark side to confidence?

For the individual, it increases reckless risk-taking: accidents, gambling, debt and fights you will lose. It also increases health problems: smoking, drinking, drug use, excessive eating. And it promotes antisocial tendencies, such as being self-centered, thinking that you’re always right, intimidating and bullying less-confident people.

For society, it promotes a narcissistic culture—where we worship people who worship themselves—and enables too many incompetent people, who, because of their confidence, manage to somehow seem more competent to the untrained observer, to rise to the top, often at the expense of people who are less confident but actually more competent.

Who would you rather pick as your pilot: someone who is confident but incompetent, or someone who is not very confident but competent? The only excuse for rewarding confidence is our inability to detect actual competence—this is not without consequences.

As I read Confidence, the Clint Eastwood line from the movie, Dirty Harry—”A man has got to know his limitations”—kept running through my mind. Care to comment?

Clint had it right. The fact that in much of the Western world, we tend to chase and promote high levels of self-belief, as opposed to high levels of self-awareness, is really quite alarming. The only advantage of not being aware of one’s limitation is that it can help us hide our limitations from others, but this is only true when others are unable to judge our expertise. The consequences for society, rather than the individual, are quite unfortunate: a world full of people who successfully pretend to be better than they really are is a world of charlatans and con-artists.

You had me at “reality distortion,” which you mention just three paragraphs into the book. It’s a term we’ve become familiar with through Walter Isaacson’s biography of Steve Jobs, the master of reality distortion. Play FBI profiler with me on Jobs through the lens of confidence.

There’s a common assumption that successful entrepreneurs owe their success to their ability to distort reality, but we only have anecdotal evidence for this, such as the Steve Jobs case. There’s also a big difference between self-deceiving biases, which distort reality to enable us to maintain high self-views and avoid hurting our self-esteem, and the ability to interpret the world in unique ways, such as seeing opportunities where others do not and translating those opportunities into profitable enterprises.

Although Steve Jobs was clearly a narcissist, his vision and talent for innovation were unquestionable. The issue in his case is not that he maintained unrealistic self-views but that he had an almost incomparable talent for seeing things before others could, not least because he was able to transform his vision into a new reality. In the words of the great Alan Kay, “The best way to predict the future is to invent it.” It takes a lot of talent to do that, and Steve Jobs had that talent.

The positive psychology movement hails focusing on your strengths and working around your weaknesses as the path to success. What’s wrong with that thinking?

I have nothing against the original positive psychology movement, which was not only well-intentioned but also interesting from an intellectual point of view. Unfortunately, this movement has been washed down to a self-indulgent and unintellectual compilation of cliches that promote a narcissistic mindset and the pursuit of confidence rather than competence. It consists of messages, such as “Don’t worry about what others think of you. You’re great no matter what they say”; “Believe in yourself, and you will achieve anything you want”; “Don’t worry about your problems—stay positive and they will go away.” So many things are wrong with this.

First, it is actually very hard to deliberately manipulate our self-views or emotions without either distorting reality or consuming mood-enhancing substances. Second, in the unlikely event of succeeding—and feeling better about ourselves—there will be no gains other than feeling good. Third, negative emotions are highly adaptive: They have evolved as a threat-detection signal, they highlight a discrepancy between where we are and where we want to be, and they make us more attentive to negative feedback, which is a key to improvement. Ironically, then, what started as a self-improvement movement has turned into a self-enhancement movement, and self-enhancement can only prevent self-improvement.

You maintain that Dale Carnegie’s 19 principles for “winning friends and influencing people” are easier to implement for people with lower rather higher confidence. Can you explain why?

Because people who are modest and insecure tend to be less self-centered, they will find it easier to pay attention to others, which, as the great Dale Carnegie pointed out, is the best way to influence them. Moreover, for people who feel less assertive in social interactions, focusing on others is a great strategy for overcoming their own anxieties—it’s distracting and effective in terms of making them more likeable.

Just like most people are overconfident, most people are only interested in themselves, which can be evidenced in 80 percent of the conversations between two people: More often than not, there are two conversations, not one, because each person is talking about what they want, ignoring the other person’s interest and wishes.

You mentioned the word “likable,” and likeability is a relevant theme in today’s social media drenched world. What’s your advice on being likeable?

Some people are naturally more likeable than others, and the most common reasons for this are that they’re more charismatic or have higher social status; for instance, they’re more attractive, connected or successful. But charisma is hugely overrated. In fact, unless you can back it up, charisma will be little more than arrogance.

For example, many psychological studies have shown that when people perceive that we have a “confidence surplus” and see ourselves more favorably than we should, they will like us less and think of us as cocky and deluded. Social media provides a great setting to study this: You only need to look at people’s updates on Facebook, which is already a narcissistic environment, to see how ineffective displays of hubris really are.

What’s the role of confidence in small-business failure?

Most entrepreneurs are overconfident, but then again, most entrepreneurs fail. There are robust studies on this, and they show very clearly that overconfidence and high risk-taking do increase the probability of starting a business, and that makes sense: Given the low chances of success, you have to be somewhat deluded and love risk in order to start a business.

However, risk-aversion and under-confidence are more likely to cause entrepreneurial success; and the key competencies that make certain entrepreneurs more successful are creativity, proactivity, opportunism and vision. Confidence is pretty much irrelevant.

Seventy percent of startups go bust in the first 10 years, and very few will ever grow or employ more than five employees. The lesson is clear: Most entrepreneurs have no talent for innovation, just like most managers have no talent for leadership. The fact that they launch a business doesn’t mean they have the capacity to make that business grow. The fact that they tend to be confident doesn’t imply that confidence is beneficial. As a matter of fact, it may well be the very reason they fail.

At least there are benefits for society: Quantity does lead to quality. In other words, if we have thousands of startups, we are more likely to see some really successful ones than if we have just a few startups. However, I’m pretty sure that venture capitalists and angel investors end up getting more ROI when they fund people who are competent rather than confident.

What’s the most dangerous myth concerning career success?

I would say that having unrealistic expectations—for instance, “I will be the next Richard Branson”—and a sense of entitlement, such as “I deserve to be the CEO of Apple,” is a recipe for disaster, not least because they’re usually coupled with a poor work ethic. In other words, there’s a big gap between most people’s aspirations and their dedication; too many people—especially millennials and Gen Y—appear to want certain things without really wanting to do what it takes to achieve them, yet we keep feeding their delusions of grandeur.

The essence of ambition is not to have outrageous goals but to work harder than anyone else to achieve whatever goals you have and to have the ability to remain always dissatisfied with what you achieve. Ambitious goals with a lazy work ethic is a killer combination. Unless people start working harder, they would be better off having more modest aspirations and realistic goals.

What’s the one thing you want readers to take away from this book?

Confidence is overrated.

If you’re interested in META profiling and wish to take the META test, you can do so here. You connect with Dr. Chamorro-Premuzic on LinkedIn and follow him on Twitter.