A Cure for Overconfidence

“FACTS FROM THE INTERNET”

• 86% of a group of college students say that they are better looking than their classmates
• 19% of people think that they belong to the richest 1% of the population
• 82% of people say they are in the top 30% of Safe Drivers
• 80% of students think they will finish in the top half of their class
• In a confidence-intervals task, where subjects had to judge quantities such as the total egg production of the U.S. or the total number of physicians and surgeons in the Boston Yellow Pages, they expected an error rate of 2% when their real error rate was 46%.
• 68% of lawyers in civil cases believe that their side will win
• 81% of new business owners think their business will succeed, but also say that 61% of the businesses like theirs will fail

But on the other hand,

• A test of 25,000 predictions by weather forecasters found no overconfidence

We all know what is different about weather forecasters.  The make predictions regularly with confidence intervals attached AND they always get feedback about how good that their forecast actually was.

So the Overconfidence effect, that is seen by psychologists as one of the most reliable of biases in decision making, is merely the effect of under training in developing opinions about confidence intervals.

This conclusion leads directly to a very important suggestion for risk managers.  Of course risk managers are trying to act like weather forecasters.  But they are often faced with an audience who are overconfident – they believe that their ability to manage the risks of the firm will result in much better outcomes than is actually likely.

But the example of weather forecasters seems to show that the ability to realistically forecast confidence intervals can be learned by a feedback process.  Risk managers should make sure that in advance of every forecast period that they make the model for frequency and severity of losses are widely known.  And then at the end of every forecast period that they show how actual experience does or does not confirm the forecast.

Many risk models allow for a prediction of the likelihood of every single exact dollar gain or loss that is seen to be possible.  So at the end of each period, when the gain or loss for that period is known, the risk manager should make a very public review of the likelihoods that were predicted for the level of gain or loss that actually occurred.

This sort of process is performed by the cat modelers.  After every major storm, they go through a very public process of discovering what the model said was the likelihood of the size loss that the storm produced.

The final step is to decide whether or not to recalibrate the model as a result of the storm.

Overconfidence can be cured by experience.

Explore posts in the same categories: Enterprise Risk Management, Risk Learning

Tags: risk assessment

You can comment below, or link to this permanent URL from your own site.