Risk Assessment is always Opinion

Risk Assessment is most often done with very high tech models.

There is a cycle for risk models though.  The cycle starts with a simple model and progresses to ever more sophisticated models.  The ability to calculate risk at any time of the day or night becomes an achievable goal.

But just as the models get to be almost perfect, something often happens.  People start to doubt the model.  Then it shifts from sporadic doubt to rampant disbelief.  Then the process reaches its final stage and the model is totally ignored and abandoned.

What is the cause of that cycle?  It is caused by the fact that the process of modeling is always built on an opinion.  But as the model gets more and more sophisticated, the modelers forget the basic opinion.  They come to feel that the sophistication makes the model a machine that is capable of producing ultimate truth.

But folks who are not involved in the modeling, who are not drawn in to the process of creating greater and greater refinement to the risk assessments, will judge the model by the degree to which it helps with managing the business.  By the results of management judgements that are informed by the models.

The models will of course be perfectly fine when they deal with events that occur within one standard deviation of the mean.  Those events happen fairly frequently and there will be plenty of data to calibrate the frequency for those events.

But that is not where the real risk is located – within one standard deviation.

Real risk is most often found at least 2 standard deviations out.

Nassim Taleb has indicated that it is important to notice that the most significant risks are always out so far in the distribution that there is never enough data to properly calibrate the model.

But Taleb would only be correct if the important information about a risk is the PAST frequency.

That is not correct.  The important thing about risks is the FUTURE frequency.  The future frequency is unknowable.

But you can have an opinion about that frequency.

  1. Your opinion could be that the future will be just like the past.
  2. Your opinion could be that the future will be worse than the past.
  3. Your opinion could be that the future will be better than the past.
  4. Your opinion could be that you do not know the future.

You may form that opinion based on the opinion that seems to be implied by the market prices, or by listening to experts.

The folks with opinion 1 tend to build the models.  They can collect the data to calibrate their models.  But the idea that the future will be just like the past is simply their OPINION.  They do not know.

The folks with opinion 2 tend to try to avoid risks.  They do not need models to do that.

The folks with opinion 3 tend to take risks that they think are overpriced from the folks with opinions 1 & 2.  Models get n their way.

The folks with opinion 4 do not believe in models.

So the people who have opinion 1 look around and see that everyone who makes models believes that the future will be just like the past and they eventually come to believe that it is the TRUTH, not just an OPINION.

They come to believe that people with opinions 2,3,4 are all misguided.

But in fact, sometimes the future is riskier than the past.  Sometimes it is less risks.  And sometimes, it is just too uncertain to tell (like right now).

And sometimes the future is just like the past.  And the models work just fine.

Advertisements
Explore posts in the same categories: Enterprise Risk Management, Modeling

Tags:

You can comment below, or link to this permanent URL from your own site.

2 Comments on “Risk Assessment is always Opinion”

  1. Andrew Howe Says:

    Absolutely outstanding, particularly given the brevity. “Plight of the fortune tellers” is another good (book) read with an arguably similar message, though somewhat longer.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: