Posted tagged ‘risk assessment’

Is it rude to ask “How fat is your tail?”

July 23, 2014

In fact, not only is it not rude, the question is central to understanding risk models.  The Coefficient of Riskiness(COR) allows us for the first time to talk about this critical question.

332px-36_Stanley_Hawk

You see, “normal” sized tails have a COR of three. If everything were normal, then risk models wouldn’t be all that important. We could just measure volatility and multiply it by 3 to get the 1 in 1000 result. If you instead want the 1 in 200 result, you would multiply the 1 in 1000 result by 83%.

Amazing maths fact – 3 is always the answer.

But everything is not normal. Everything does not have a COR of 3. So how fat are your tails?

RISKVIEWS looked at an equity index model. That model was carefully calibrated to match up with very long term index returns (using Robert Shiller’s database). The fat tailed result there has a COR of 3.5. With that model the 2008 S&P 500 total return loss of 37% is a 1 in 100 loss.

So if we take that COR of 3.5 and apply it to the experience of 1971 to 2013 that happens to be handy, the mean return is 12% and the volatility is about 18%. Using the simple COR approach, we estimate the 1 in 1000 loss as 50% (3.5 times the volatility subtracted from the average). To get the 1/200 loss, we can take 83% of that and we get a 42% loss.

RISKVIEWS suggests that the COR can be an important part of Model Validation.

 Looking at the results above for the stock index model, the question becomes why is 3.5 then the correct COR for the index? We know that in 2008, the stock market actually dropped 50% from high point to low point within a 12 month period that was not a calendar year. If we go back to Shiller’s database, which actually tracks the index values monthly (with extensions estimated for 50 years before the actual index was first defined), we find that there are approximately 1500 12 month periods. RISKVIEWS recognizes that these are not independent observations, but to answer this particular question, these actually are the right data points. And looking at that data, a 50% drop in a 12 month period is around the 1000th worst 12 month period. So a model with a 3.5 COR is pretty close to an exact fit with the historical record. And what if you have an opinion about the future riskiness of the stock market? You can vary the volatility assumptions if you think that the current market with high speed trading and globally instantaneously interlinked markets will be more volatile than the past 130 years that Schiller’s data covers. You can also adjust the future mean. You might at least want to replace the historic geometric mean of 10.6% for the arithmetic mean quoted above of 12% since we are not really taking about holding stocks for just one year. And you can have an opinion about the Riskiness of stocks in the future. A COR of 3.5 means that the tail at the 1 in 1000 point is 3.5 / 3 or 116.6% of the normal tails. That is hardly an obese tail.

The equity index model that we started with here has a 1 in 100 loss value of 37%. That was the 2008 calendar total return for the S&P 500. If we want to know what we would get with tails that are twice as fat, with the concept of COR, we can look at a COR of 4.0 instead of 3.5. That would put the 1 in 1000 loss at 9% worse or 59%. That would make the 1 in 200 loss 7% worse or 49%.

Those answers are not exact. But they are reasonable estimates that could be used in a validation process.

Non-technical management can look at the COR for each model can participate in a discussion of the reasonability of the fat in the tails for each and every risk.

RISKVIEWS believes that the COR can provide a basis for that discussion. It can be like the Richter scale for earthquakes or the Saffir-Simpson scale for hurricanes. Even though people in general do not know the science underlying either scale, they do believe that they understand what the scale means in terms of severity of experience. With exposure, the COR can take that place for risk models.

Chicken Little or Coefficient of Riskiness (COR)

July 21, 2014

Running around waving your arms and screaming “the Sky is Falling” is one way to communicate risk positions.  But as the story goes, it is not a particularly effective approach.  The classic story lays the blame on the lack of perspective on the part of Chicken Little.  But the way that the story is told suggests that in general people have almost zero tolerance for information about risk – they only want to hear from Chicken Little about certainties.

But insurers live in the world of risk.  Each insurer has their own complex stew of risks.  Their riskiness is a matter of extreme concern.  Many insurers use complex models to assess their riskiness.  But in some cases, there is a war for the hearts and minds of the decision makers in the insurer.  It is a war between the traditional qualitative gut view of riskiness and the new quantitative view of riskiness.  One tactic in that war used by the qualitative camp is to paint the quantitative camp as Chicken Little.

In a recent post, Riskviews told of a scale, a Coefficient of Riskiness.  The idea of the COR is to provide a simple basis for taking the argument about riskiness from the name calling stage to an actual discussion about Riskiness.

For each risk, we usually have some observations.  And from those observations, we can form the two basic statistical facts, the observed average and observed volatility (known as standard deviation to the quants).  But in the past 15 years, the discussion about risk has shifted away from the observable aspects of risk to an estimate of the amount of capital needed for each risk.

Now, if each risk held by an insurer could be subdivided into a large number of small risks that are similar in riskiness for each (including size of potential loss) and where the reasons for the losses for each individual risk were statistically separate (independent) then the maximum likely loss to be expected (99.9%tile) would be something like the average loss plus three times the volatility.  It does not matter what number is the average or what number is the standard deviation.

RISKVIEWS has suggested that this multiple of 3 would represent a standard amount of riskiness and become the index value for the Coefficient of Riskiness.

This could also be a starting point in looking at the amount of capital needed for any risks.  Three times the observed volatility plus the observed average loss.  (For the quants, this assumes that losses are positive values and gains negative.  If you want losses to be negative values, then take the observed average loss and subtract three times the volatility).

So in the debate about risk capital, that value is the starting point, the minimum to be expected.  So if a risk is viewed as made up of substantially similar but totally separate smaller risks (homogeneous and independent), then we start with a maximum likely loss of average plus three times volatility.  Many insurers choose (or have chosen for them) to hold capital for a loss at the 1 in 200 level.  That means holding capital for 83% of this Maximum Likely Loss.  This is the Viable capital level.  Some insurers who wish to be at the Robust level of capital will hold capital roughly 10% higher than the Maximum Likely Loss.  Insurers targeting the Secure capital level will hold capital at approximately 100% of the Maximum Likely Loss level.

But that is not the end of the discussion of capital.  Many of the portfolios of risks held by an insurer are not so well behaved.  Those portfolios are not similar and separate.  They are dissimilar in the likelihood of loss for individual exposures, they are dissimilar for the possible amount of loss.  One way of looking at those dissimilarities is that the variability of rate and of size result in a larger number of pooled risks acting statistically more like a smaller number of similar risks.

So if we can imagine that evaluation of riskiness can be transformed into a problem of translating a block of somewhat dissimilar, somewhat interdependent risks into a pool of similar, independent risks, this riskiness question comes clearly into focus.  Now we can use a binomial distribution to look at riskiness.  The plot below takes up one such analysis for a risk with an average incidence of 1 in 1000.  You see that for up to 1000 of these risks, the COR is 5 or higher.  The COR gets up to 6 for a pool of only 100 risks.  It gets close to 9 for a pool of only 50 risks.

 

cor

 

There is a different story for a risk with average incidence of 1 in 100.  COR is less than 6 for a pool as small as 25 exposures and the COR gets down to as low as 3.5.

Cor100

In producing these graphs, RISKVIEW notices that COR is largely a function of number of expected claims.  So The following graph shows COR plotted against number of expected claims for low expected number of claims.  (High expected claims produces COR that is very close to 3 so are not very interesting.)

COR4You see that the COR stays below 4.5 for expected claims 1 or greater.  And there does seem to be a gently sloping trend connecting the number of expected claims and the COR.

So for risks where losses are expected every year, the maximum COR seems to be under 4.5.  When we look at risks where the losses are expected less frequently, the COR can get much higher.  Values of COR above 5 start showing up with expected losses that are in the range of .2 and values above .1 are even higher.

cor5

What sorts of things fit with this frequency?  Major hurricanes in a particular zone, earthquakes, major credit losses all have expected frequencies of one every several years.

So what has this told us?  It has told us that fat tails can come from the small portfolio effect.  For a large portfolio of similar and separate risks, the tails are highly likely to be normal with a COR of 3.  For risks with a small number of exposures, the COR, and therefore the tail, might get as much as 50% fatter with a COR of up to 4.5. And the COR goes up as the number of expected losses goes down.

Risks with very fat tails are those with expected losses less frequent than one per year can have much fatter tails, up to three times as fat as normal.

So when faced with those infrequent risks, the Chicken Little approach is perhaps a reasonable approximation of the riskiness, if not a good indicator of the likelihood of an actual impending loss.

 

Quantitative vs. Qualitative Risk Assessment

July 14, 2014

There are two ways to assess risk.  Quantitative and Qualitative.  But when those two words are used in the NAIC ORSA Guidance Manual, their meaning is a little tricky.

In general, one might think that a quantitative assessment uses numbers and a qualitative assessment does not.  The difference is as simple as that.  The result of a quantitative assessment would be a number such as $53 million.  The result of a qualitative assessment would be words, such as “very risky” or “moderately risky”.

But that straightforward approach to the meaning of those words does not really fit with how they are used by the NAIC.  The ORSA Guidance Manual suggests that an insurer needs to include those qualitative risk assessments in its determination of capital adequacy.  Well, that just will not work if you have four risks that total $400 million and three others that are two “very riskys” and one “not so risk”.  How much capital is enough for two “very riskys”, perhaps you need a qualitative amount of surplus to provide for that, something like “a good amount”.

RISKVIEWS believes that then the NAIC says “Quantitative” and “Qualitative” they mean to describe two approaches to developing a quantity.  For ease, we will call these two approaches Q1 and Q2.

The Q1 approach is data and analysis driven approach to developing the quantity of loss that the company’s capital standard provides for.  It is interesting to RISKVIEWS that very few participants or observers of this risk quantification regularly recognize that this process has a major step that is much less quantitative and scientific than others.

The Q1 approach starts and ends with numbers and has mathematical steps in between.  But the most significant step in the process is largely judgmental.  So at its heart, the “quantitative” approach is “qualitative”.  That step is the choice of mathematical model that is used to extrapolate and interpolate between actual data points.  In some cases, there are enough data points that the choice of model can be based upon somewhat less subjective fit criteria.  But in other cases, that level of data is reached by shortening the time step for observations and THEN making heroic (and totally subjective) assumptions about the relationship between successive time periods.

These subjective decisions are all made to enable the modelers to make a connection between the middle of the distribution, where there usually is enough data to reliably model outcomes and the tail, particularly the adverse tail of the distribution where the risk calculations actually take place and where there is rarely if ever any data.

There are only a couple of subjective decisions possibilities, in broad terms…

  • Benign – Adverse outcomes are about as likely as average outcomes and are only moderately more severe.
  • Moderate – Outcomes similar to the average are much more likely than outcomes significantly different from average.  Outcomes significantly higher than average are possible, but likelihood of extremely adverse outcomes are extremely highly unlikely.
  • Highly risky – Small and moderately adverse outcomes are highly likely while extremely adverse outcomes are possible, but fairly unlikely.

The first category of assumption, Benign,  is appropriate for large aggregations of small loss events where contagion is impossible.  Phenomenon that fall into this category are usually not the concern for risk analysis.  These phenomenon are never subject to any contagion.

The second category, Moderate, is appropriate for moderate sized aggregations of large loss events.  Within this class, there are two possibilities:  Low or no contagion and moderate to high contagion.  The math is much simpler if no contagion is assumed.

But unfortunately, for risks that include any significant amount of human choice, contagion has been observed.  And this contagion has been variable and unpredictable.  Even more unfortunately, the contagion has a major impact on risks at both ends of the spectrum.  When past history suggests a favorable trend, human contagion has a strong tendency to over play that trend.  This process is called “bubbles”.  When past history suggests an unfavorable trend, human contagion also over plays the trend and markets for risks crash.

The modelers who wanted to use the zero contagion models, call this “Fat Tails”.  It is seen to be an unusual model, only because it was so common to use the zero contagion model with the simpler maths.

RISKVIEWS suggests that when communicating that the  approach to modeling is to use the Moderate model, the degree of contagion assumed should be specified and an assumption of zero contagion should be accompanied with a disclaimer that past experience has proven this assumption to be highly inaccurate when applied to situations that include humans and therefore seriously understates potential risk.

The Highly Risky models are appropriate for risks where large losses are possible but highly infrequent.  This applies to insurance losses due to major earthquakes, for example.  And with a little reflection, you will notice that this is nothing more than a Benign risk with occasional high contagion.  The complex models that are used to forecast the distribution of potential losses for these risks, the natural catastrophe models go through one step to predict possible extreme events and the second step to calculate an event specific degree of contagion for an insurer’s specific set of coverages.

So it just happens that in a Moderate model, the 1 in 1000 year loss is about 3 standard deviations worse than the mean.  So if we use that 1 in 1000 year loss as a multiple of standard deviations, we can easily talk about a simple scale for riskiness of a model:

Scale

So in the end the choice is to insert an opinion about the steepness of the ramp up between the mean and an extreme loss in terms of multiples of the standard deviation.  Where standard deviation is a measure of the average spread of the observed data.  This is a discussion that on these terms include all of top management and the conclusions can be reviewed and approved by the board with the use of this simple scale.  There will need to be an educational step, which can be largely in terms of placing existing models on the scale.  People are quite used to working with a Richter Scale for earthquakes.  This is nothing more than a similar scale for risks.  But in addition to being descriptive and understandable, once agreed, it can be directly tied to models, so that the models are REALLY working from broadly agreed upon assumptions.

*                  *                *               *             *                *

So now we go the “Qualitative” determination of the risk value.  Looking at the above discussion, RISKVIEWS would suggest that we are generally talking about situations where we for some reason do not think that we know enough to actually know the standard deviation.  Perhaps this is a phenomenon that has never happened, so that the past standard deviation is zero.  So we cannot use the multiple of standard deviation method discussed above.  Or to put is another way, we can use the above method, but we have to use judgment to estimate the standard deviation.

*                  *                *               *             *                *

So in the end, with a Q1 “quantitative” approach, we have a historical standard deviation and we use judgment to decide how risky things are in the extreme compared to that value.  In the Q2 “qualitative” approach, we do not have a reliable historical standard deviation and we need to use judgment to decide how risky things are in the extreme.

Not as much difference as one might have guessed!

What if there are no clocks?

March 17, 2014

RISKVIEWS recently told someone that the idea of a Risk Control Cycle was quite simple.  In fact, it is just as simple as making an appointment and keeping it.

But what if you are in a culture that has no clocks?

Bahnsteiguhr[1]

Imagine how difficult the conversation might be about an appointment for 9:25 tomorrow morning.

That is the situation for companies who want to learn about adopting a risk control cycle who have no tradition of measuring risk.

The companies who have dutifully followed a regulatory imperative to install a capital model may think that they have a risk measurement system.  But that system is like a clock that they only look at once per month.  Not very helpful for making and keeping appointments.

Risk control needs to be done with risk measures that are available frequently.  That probably will mean that the risk measure that is most useful for risk control might not be as spectacularly accurate as a capital model.  The risk control process needs a quick measure of risk that can be available every week or at least every month.  Information at the speed of your business decision making process.

But none of us are really in a culture where there are no clocks.  Instead, we are in cultures where we choose not to put any clocks up on the walls.  We choose not to set times for our appointments.

I found that if you have a goal, that you might not reach it. But if you don’t have one, then you are never disappointed. And I gotta tell ya… it feels phenomenal.

from the movie Dodgeball

Stress to reduce Distress

February 12, 2014

Distress lurks. Just out of sight. Perhaps around a corner, perhaps down the road just past your view.
For some their rule is “out of sight, out of mind”. For them, worry and preparation can start when, and if, distress comes into sight.

But risk managers see it as our jobs to look for and prepare for distress. Whether it is in sight or not. Especially because some sorts of distress come on so very quickly and some methods of mitigation take effect so slowly.
Stress testing is one of the most effective tools, both for imagining the potential magnitude of distresses, but almost more importantly, in developing compelling stories to communicate about that distress potential.

This week, Willis Wire is featuring a piece about Stress Testing in the “ERM Practices” series;

ERM Practices:  Stress Testing

RISKVIEWS has features many posts related to Stress Testing:

RISKVIEWS Archive of Posts related to Stress Testing

You need good Risk Sense to run an insurance company

January 16, 2014

It seems to happen all too frequently.

A company experiences a bad loss and the response of management is that they were not aware that the company had such a risk exposure.

For an insurance company, that response just isn’t good enough.  And most of the companies where management has given that sort of answer were not insurers.

At an insurance company, managers all need to have a good Risk Sense.

Risk Sense is a good first order estimate of the riskiness of all of their activities. 

Some of the companies who have resisted spending the time, effort and money to build good risk models are the companies whose management already has an excellent Risk Sense.  Management does not see the return for spending all that is required to get what is usually just the second digit.

By the way, if you think that your risk model provides reliable information beyond that second digit, you need to spend more time on model validation.

To have a reliable Risk Sense, you need to have reliable risk selection and risk mitigation processes.  You need to have some fundamental understanding of the risks that are out there in the areas in which you do business.  You also need to  be constantly vigilant about changes to the risk environment that will require you to adjust your perception of risk as well as your risk selection and mitigation practices.

Risk Sense is not at all a “gut feel” for the risk.  It is instead more of a refined heuristic.  (See Evolution of Thinking.)  The person with Risk Sense has the experience and knowledge to fairly accurately assess risk based upon the few really important facts about the risks that they need to get to a conclusion.

The company that needs a model to do basic risk assessment, i.e. that does not have executives who have a Risk Sense, can be highly fragile.  That is because risk models can be highly fragile.  Good model building actually requires plenty of risk sense.

The JP Morgan Chase experiences with the “London Whale” were a case of little Risk Sense and staff who exploited that weakness to try to get away with excessive risk taking.  They relied completely on a model to tell them how much risk that they were taking.  No one looked at the volume of activity and had a usual way to create a good first order estimate of the risk.  The model that they were using was either inaccurate for the actual situation that they were faced with or else it was itself gamed.

A risk management system does not need to work quite so hard when executives have a reliable Risk Sense.  If an executive can look at an activity report and apply their well honed risk heuristics, they can be immediately informed of whether there is an inappropriate risk build up or not.  They need control processes that will make sure that the risk per unit of activity is within regular bounds.  If they start to have approved activities that involve situations with much higher levels of risk per unit of activity, then their activity reports need to separate out the more risky activities.

Models are too fragile to be the primary guide to the level of risk.  Risk taking organizations like insurers need Risk Sense.

Can’t skip measuring Risk and still call it ERM

January 15, 2014

Many insurers are pushing ahead with ERM at the urging of new executives, boards, rating agencies and regulators.  Few of those firms who have resisted ERM for many years have a history of measuring most of their risks.

But ERM is not one of those liberal arts like the study of English Literature.  In Eng Lit, you may set up literature classification schemes, read materials, organize discussion groups and write papers.  ERM can have those elements, but the heart of ERM is Risk Measurement.  Comparing those risk measures to expectations and to prior period measures.  If a company does not have Risk Measurement, then they do not have ERM.

That is the tough side of this discussion, the other side is that there are many ways to measure risks and most companies can implement several of them for each risk without the need for massive projects.

Here are a few of those measures, listed in order of increasing sophistication:

1. Risk Guesses (AKA Qualitative Risk Assessment)
– Guesses, feelings
– Behavioral Economics Biases
2. Key Risk Indicators (KRI)
– Risk is likely to be similar to …
3. Standard Factors
– AM Best,  S&P, RBC
4. Historical Analysis
– Worst Loss in past 10 years as pct of base (premiums,assets).
5. Stress Tests
– Potential loss from historical or hypothetical scenario
6. Risk Models
– If the future is like the past …
– Or if the future is different from the past in this way …

More discussion of Risk Measurement on WillisWire:

     Part 2 of a 14 part series
And on RISKVIEWS:
Risk Assessment –  55 other posts relating to risk measurement and risk assessment.

Follow

Get every new post delivered to your Inbox.

Join 658 other followers

%d bloggers like this: