Why do so many investors attempt to beat the market each year when research clearly shows their odds of success are extremely low?

Nobel Laureate Daniel Kahneman, the only psychologist ever to win the Nobel Prize in Economics, says human behavioral biases are to blame. Kahneman’s research reveals people are naturally overconfident. Survey results routinely illustrate this tendency—most respondents rate themselves as above-average drivers (an obvious statistical impossibility).

The same bias applies to the investment world. Duke’s Fuqua School of Business has conducted a quarterly poll of Chief Financial Officers (CFOs) since 1996. The Duke CFO Magazine survey[1] asks hundreds of CFOs from the country’s top companies for forecasts on a range of economic issues, including future stock returns. One can easily imagine how CFOs, those closest to the financial pulse of American business, could feel self-assured gauging the prospects of individual companies and the economy at large. Their results, however, undermine that confidence.

As it turns out, these CFOs have no idea what the market is going to do—the correlation between their market forecasts and realized returns is actually negative (markets tend to move in the opposite direction of their forecasts). What’s most alarming is the confidence with which they make such inaccurate forecasts.

The CFOs are asked to provide a range of expected returns wide enough that actual results will fall within the range 80% of the time. According to Kahneman, the executives’ overconfidence causes them to make their ranges much too narrow.[2] The result is actual returns fall within their predicted range just 33% of the time. “They have no idea,” says Kahneman, but “they don’t know it.”

The lesson? Adding an occasional slice of humble pie to your diet is good for your financial health.
[1] Itzhak Ben-David, John Graham and Campbell Harvey. “Managerial Miscalibration.” July 2010.
[2] Housel, Morgan. “An interview with Dr. Daniel Kahneman.” June 28, 2013.