Preparing for the Zombie Apocalypse

The CDC now has a page with preparedness tips for the next Zombie Apocalypse.

If you read that closely, you might notice that the preparedness tips are exactly the same as their tips for Hurricanes or Pandemics.

So maybe this is a good way to get folks to pay attention to disaster preparedness?

Must be better than the way that that some office buildings make preparedness into a mind numbing drill that is certain to take the edge off of any possible hint of preparedness.

Perhaps a suggestion for your next fire drill – have zombies show up and find out how many people were ready and how many got eaten by the zombies.

What’s Next?

Turbulent Times are Next.

At BusinessInsider.com, a feature from Guillermo Felices tells of 8 shocks that are about to slam the global economy.

#1 Higher Food Prices in Emerging Markets

#2 Higher Interest Rates and Tighter Money in Emerging Markets

#3 Political Crises in the Middle East

#4 Surging Oil Prices

#5 An Increase in Interest Rates in Developed Markets

#6 The End of QE2

#7 Fiscal Cuts and Sovereign Debt Crises

#8 The Japanese Disaster

How should ideas like these impact on ERM systems?  Is it at all reasonable to say that they should not? Definitely not.

These potential shocks illustrate the need for the ERM system to be reflexive.  The system needs to react to changes in the risk environment.  That would mean that it needs to reflect differences in the risk environment in three possible ways:

1. In the calibration of the risk model.  Model assumptions can be adjusted to reflect the potential near term impact of the shocks.  Some of the shocks are certain and could be thought to impact on expected economic activity (Japanese disaster) but have a range of possible consequences (changing volatility).  Other shocks, which are much less certain (end of QE2 – because there could still be a QE3) may be difficult to work into model assumptions.
2. With Stress and Scenario Tests – each of these shocks as well as combinations of the shocks could be stress or scenario tests.  Riskviews suggest that developing a handful of fully developed scenarios with 3 or more of these shocks in each would be the modst useful.
3. In the choices of Risk Appetite.  The information and stress.scenario tests should lead to a serious reexamination of risk appetite.  There are several reasonable reactions – to simply reduce risk appetite in total, to selectively reduce risk appetite, to increase efforts to diversify risks, or to plan to aggressively take on more risk as some risks are found to have much higher reward.

The last strategy mentioned above (aggressively take on more risk) might not be thought of by most to be a risk management strategy.  But think of it this way, the strategy could be stated as an increase in the minimum target reward for risk.  Since things are expected to be riskier, the firm decides that it must get paid more for risk taking, staying away from lower paid risks.  This actually makes quite a bit MORE sense than taking the same risks, expecting the same reward for risks and just taking less risk, which might be the most common strategy selected.

The final consideration is compensation.  How should the firm be paying people for their performance in a riskier environment?  How should the increase in market risk premium be treated?

See Risk adjusted performance measures for starters.

More discussion on a future post.

Sins of Risk Measurement

.

Read The Seven Deadly Sins of Measurement by Jim Campy

Measuring risk means walking a thin line.  Balancing what is highly unlikely from what it totally impossible.  Financial institutions need to be prepared for the highly unlikely but must avoid getting sucked into wasting time worrying about the totally impossible.

Here are some sins that are sometimes committed by risk measurers:

1.  Downplaying uncertainty.  Risk measurement will always become more and more uncertain with increasing size of the potential loss numbers.  In other words, the larger the potential loss, the less certain you can be about how certain it might be.  Downplaying uncertainty is usually a sin of omission.  It is just not mentioned.  Risk managers are lured into this sin by the simple fact that the less that they mention uncertainty, the more credibility their work will be given.

2.  Comparing incomparables.  In many risk measurement efforts, values are developed for a wide variety of risks and then aggregated.  Eventually, they are disaggregated and compared.  Each of the risk measurements are implicitly treated as if they were all calculated totally consistently.  However,  in fact, we are usually adding together measurements that were done with totally different amounts of historical data, for markets that have totally different degrees of stability and using tools that have totally different degrees of certitude built into them.  In the end, this will encourage decisions to take on whatever risks that we underestimate the most through this process.

3.  Validate to Confirmation.  When we validate risk models, it is common to stop the validation process when we have evidence that our initial calculation is correct.  What that sometimes means is that one validation is attempted and if validation fails, the process is revised and tried again.  This is repeated until the tester is either exhausted or gets positive results.  We are biased to finding that our risk measurements are correct and are willing to settle for validations that confirm our bias.

4.  Selective Parameterization.  There are no general rules for parameterization.  Generally, someone must choose what set of data is used to develop the risk model parameters.  In most cases, this choice determines the answers of the risk measurement.  If data from a benign period is used, then the measures of risk will be low.  If data from an adverse period is used, then risk measures will be high.  Selective paramaterization means that the period is chosen because the experience was good or bad to deliberately influence the outcome.

5.  Hiding behind Math.  Measuring risk can only mean measuring a future unknown contingency.  No amount of fancy math can change that fact.  But many who are involved in risk measurement will avoid ever using plain language to talk about what they are doing, preferring to hide in a thicket of mathematical jargon.  Real understanding of what one is doing with a risk measurement process includes the ability to say what that entails to someone without an advanced quant degree.

6.  Ignoring consequences.  There is a stream of thinking that science can be disassociated from its consequences.  Whether or not that is true, risk measurement cannot.  The person doing the risk measurement must be aware of the consequences of their findings and anticipate what might happen if management truly believes the measurements and acts upon them.

7.  Crying Wolf.  Risk measurement requires a concentration on the negative side of potential outcomes.  Many in risk management keep trying to tie the idea of “risk” to both upsides and downsides.  They have it partly right.  Risk is a word that means what it means, and the common meaning associated risk with downside potential.  However, the risk manager who does not keep in mind that their risk calculations are also associated with potential gains will be thought to be a total Cassandra and will lose all attention.  This is one of the reasons why scenario and stress tests are difficult to use.  One set of people will prepare the downside story and another set the upside story.  Decisions become a tug of war between opposing points of view, when in fact both points of view are correct.

There are doubtless many more possible sins.  Feel free to add your favorites in the comments.

But one final thought.  Calling it MEASUREMENT might be the greatest sin.

Momentum Risk

How many times have you heard this

If it isn’t broken don’t fix it.

As a risk manager, momentum risk is one of the most difficult risk to overcome.  (I wonder how many times on these posts I have claimed this?)

But this is the aspect of the Horizon disaster that led to millions and millions of barrels of oil spilling into the Gulf.  Before that the oil companies claimed that there had never been a failure of an oil rig in the Gulf.  So that was the Momentum assumption.  It had never failed so it never would fail.

Standing against that is the seemly endlessly negative point of view of the risk manager:

If anything can go wrong, it will.

Murphy‘s Law is usually taken as the ultimate statement of negative pessimism.  But instead you the risk manager need to use Murphy’s law as he did.  As a mantra to keep repeating to yourself as you look for ways to stress test a system.

Looking to engineering (Murphy was an engineer you know) for some thinking about stress to failure, we find this post:

When a component is subject to increasing loads it eventually fails.   It is comparatively easy to determine the point of failure of a component subject to a single tensile force. The strength data on the material identifies this strength.   However when the material is subject to a number of loads in different directions some of which are tensile and some of which are shear, then the determination of the point of failure is more complicated…

Some of your stress to failure tests will have to be tensile, some compressive, some shear, in different directions and in different combinations.  You should do this sort of testing to know the weakest points of your system.

But there is no guarantee that the system will fail at the weakest points either.  In fact, you may put in place methods to reduce stresses to those weakest points.  Remember that now elevates other points to be the new stresses.

And do not let Momentum thinking define your approach to likelihood of these stresses.  In physical systems, the engineer knows how the system is supposed to be used and can plan for the stresses of those uses.  But in many cases, the systems designed and tested by engineers are not used in the conditions planned for or even for the exact uses that the engineer anticipated.

Sound familiar?

Human systems are not so fixed as physical systems.  Humans react to the system that they are experiencing and adjust their actions according to the feedback that they are receiving from the system.  So human systems will almost always change as they are used.

Human systems will almost always change as they are used.

That is what makes it so much more difficult to be a risk manager for a financial firm than for a firm that deals mainly with physical risks.  As noted above the humans that interface with the physical risks system do change and adapt, but there are usually a larger portion of possibilities that are fixed by the constraints of the physical systems.

With financial risks, the idea of adapting and using a type of transaction or financial structure for alternate purposes has become the occupation of a large number of folks who command a large amount of resources.

So if, for example, you are using a particular type of derivative to accomplish a fairly straightforward risk management purpose, it is quite possible that the market for that instrument will suddenly be taken over by folks with lots and lots of money, fast computers and turnover averages in the thousands per week.  Their entry into a market will change pricing and the speed of changes in pricing and then one day, suddenly, they will decide, perhaps little by little, but possibly all at once, to abandon that trade and the market will snap to being something different still.

The same sort of thing happens in insurance, but at a different speed.  Lawyers are always out there looking to “perfect” an argument to create a new class of claimants against different businesses and their insurers. THis results in a sudden jump in claims costs.

Interestingly, the strategies for those two examples might be the exact opposite.  It might be best to move on from the market that is suddenly overtaken by high speed hedge fund traders.  But the only way to recover extra losses from a newly discovered and “perfected” cause of tort is to stay with the coverage.

But in all cases, the risk manager is faced with the problem of overcoming Momentum Risk.  Convincing others that something that is not broken needs attention and possibly even fixing.

Regime Change

If something happens more or less the same way for any extended period of time, the normal reaction of humans is consider that phenomena as constant and to largely filter it out.  We do not then even try to capture new information about changes to that phenomena because our senses tell us that that input is “pure noise” with no signal.  Hence the famous story about boiling frogs.  Which may or may not be actually true about frogs, but it definitely reveals something about the way that humans take in information about the world.

But things can and do actually change.  Even things that are more or less the same for a very long time.

In the book, “This Time It’s Different”, the authors state that

“The median inflation rates before World War I were well below those of the more recent period: 0.5% per annum for 1500 – 1799 and 0.71% for 1800 – 1913, in contrast with 5% for 1914 – 2006.”

Imagine that.  Inflation averaged below 0.75% for about 300 years.  Since there is no history of extended periods of negative inflation, to get an average that low, there must be a very low standard deviation as well.  Inflation at a level of 3 or 4% is probably a one in a million situation.  Or so intelligent financial analysts before WWI must have thought that they could make plans without any concern for inflation.

But in the years following WWI, governments found a new way to default on their debts, especially their internal debts.  Reinhart and Rogoff point out that almost all of the discussion by economists regarding sovereign default is about external debt.  But they show that internal debt is very important to the situations of sovereign defaults.  Countries with high levels of internal debt and low external debt will usually not default, but countries with high levels of both internal and external debt will often default.

So as we contemplate the future of the aging western economies, we need to be careful that we do not exclude the regime changes that could occur.  And which regime changes that we should be concerned about becomes clearer when we look at all of the entitlements to retirees as debt (is there any effective difference between debt and these obligations?).  When we do that we see that there are quite a few western nations with very, very large internal debt.  And many of those countries have indexed much of that debt, taking the inflation option off of the table.

Reinhart and Rogoff also point out the sovereign default is usually not about ability to pay, it is about willingness to make the sacrifices that repayment of debt would entail.

So Risk Managers need to think about possible drastic regime changes, in addition to the seemingly highly unlikely scenario that the future will be more or less like the past.

It’s Usually the Second Truck

In many cases, companies survive the first bout of adversity.

It is the second bout that kills.

And more often than not, we are totally unprepared for that second hit.

Totally unprepared because of how we misunderstand statistics.

First of all, we believe that large loss events are unlikely and two large loss events are extremely unlikely.  So we decide not to prepare for the extremely unlikely event that we get hit by two large losses at the same time.  And in this case, “at the same time” may mean in subsequent years.  Some who look at correlation, only use an arbitrary calendar year split out of experience data.  So that they would consider losses in the third and fourth quarter to be happening at the same time but fourth quarter and first quarter of the next year would be considered different periods and therefore might show low correlations!

Second, we fail to deal with our reduced capacity immediately after a major loss event.  We still think of our capacity as it was before the first hit.  A part of our risk management plans for a major loss event should have been to immediately initiate a process to rationalize our risk exposures with our newly reduced capacity.  This may in part be due to the third issue.

Third, we misunderstand that the fact of the first event does not reduce the likelihood of the other risk events.  Those joint probabilities that made the dual event, no longer apply.  In fact, with the reduced capacity, the type of even that would incapacitate the firm has suddenly become much more likely.

Most companies that experience one large loss event do not experience a second shortly thereafter, but many companies that fail do.

So if your interest is to reduce the likelihood of failure, you should consider the two loss event situation as a scenario that you prepare for.

But those preparations will present a troubling alternative.  If, after the first major loss event, the actions needed include a sharp reduction in retained risk position, that will severely reduce the likelihood of growing back capacity.

Management is faced with a dilemma – that is two choices, neither of which are desirable.   But as with most issues in risk management, better to face those issues in advance and to make a reasoned plan, rather than looking away and hoping for the best.

But on further reflection, this issue can be seen to be one of over concentration in a single risk.  Some firms have reacted to this whole idea by setting their risk tolerance such that any one loss event will be limited to their excess capital.  Their primary strategy for this type of concentration risk is in effect a diversification strategy.

Common Terms for Severity

In the US, firms are required to disclose their risks.  This has led to an exercize that is particularly useless.  Firms obviously spend very little time on what they publish under this part of their financial statement.  Most firms seem to be using boilerplate language and a list of risks that is as long as possible.  It is clearly a totally compliance based CYA activity.  The only way that a firm could “lose” under this system is if they fail to disclose something that later creates a major loss.  So best to mention everything under the sun.  But when you look across a sector at these lists, you find a startling degree to which the risks actually differ.  That is because there is absolutely no standard that is applied to tell what is a risk and if something is a risk, how significant is it.  The idea of risk severity is totally missing.  

Bread Box

 

What would help would be a common set of terms for Severity of losses from risks.  Here is a suggestion of a scale for discussing loss severity for an individual firm: 

1. A Loss that is a threat to earnings.  This level of risk could result in a loss that would seriously impair or eliminate earnings. 
2. A Loss that could result in a significant reduction to capital.  This level of risk would result in a loss that would eliminate earnings and in addition eat into capital, reducing it by 10% to 20%
3. A Loss that could result in severe reduction of business activity.  For insurers, this would be called “Going into Run-off”.  It means that the firm is not insolvent, but it is unable to continue doing new business.  This state often lasts for several years as old liabilities of the insurer are slowly paid of as they become due.  Usually the firm in this state has some capital, but not enough to make any customers comfortable trusting them for future risks. 
4. A Loss that would result in the insolvency of the firm. 

Then in addition, for an entire sector or sub sector of firms: 

1. Losses that significantly reduce earnings of the sector.  A few firms might have capital reductions.
2. Losses that significantly impair capital for the sector.  A few firms might be run out of business from these losses.
3. Losses that could cause a significant number of firms in the sector to be run out of business.  The remainder of the sector still has capacity to pick up the business of the firms that go into run-off.  A few firms might be insolvent. 
4. Losses that are large enough that the sector no longer has the capacity to do the business that it had been doing.  There is a forced reduction in activity in the sector until capacity can be replaced, either internally or from outside funds.  A large number of firms are either insolvent or will need to go into run-off. 

These can be referred to as Class 1, Class 2, Class 3, Class 4 risks for a firm or for a sector.  

Class 3 and Class 4 Sector risks are Systemic Risks.  

Care should be taken to make sure that everyone understands that risk drivers such as equity markets, or CDS can possibly produce Class 1, Class 2, Class 3 or Class 4 losses for a firm or for a sector in a severe enough scenario.  There is no such thing as classifying a risk as always falling into one Class.  However, it is possible that at a point in time, a risk may be small enough that it cannot produce a loss that is more than a Class 1 event.  

For example, at a point in time (perhaps 2001), US sub prime mortgages were not a large enough class to rise above a Class 1 loss for any firms except those whose sole business was in that area.  By 2007, Sub Prime mortgage exposure was large enough that Class 4 losses were created for the banking sector.  

Looking at Sub Prime mortgage exposure in 2006, a bank should have been able to determine that sub primes could create a Class 1, Class 2, Class 3 or even Class 4 loss in the future.  The banks could have determined the situations that would have led to losses in each Class for their firm and determined the likelihood of each situation, as well as the degree of preparation needed for the situation.  This activity would have shown the startling growth of the sub prime mortgage exposure from a Class 1 to a Class 2 through Class 3 to Class 4 in a very short time period.  

Similarly, the prudential regulators could theoretically have done the same activity at the sector level.  Only in theory, because the banking regulators do not at this time collect the information needed to do such an exercize.  There is a proposal that is part of the financial regulation legislation to collect such information.  See CE_NIF.

The Future of Risk Management – Conference at NYU November 2009

Some good and not so good parts to this conference.  Hosted by Courant Institute of Mathematical Sciences, it was surprisingly non-quant.  In fact several of the speakers, obviously with no idea of what the other speakers were doing said that they were going to give some relief from the quant stuff.

Sad to say, the only suggestion that anyone had to do anything “different” was to do more stress testing.  Not exactly, or even slightly, a new idea.  So if this is the future of risk management, no one should expect any significant future contributions from the field.

There was much good discussion, but almost all of it was about the past of risk management, primarily the very recent past.

Here are some comments from the presenters:

• Banks need regulator to require Stress tests so that they will be taken seriously.
• Most banks did stress tests that were far from extreme risk scenarios, extreme risk scenarios would not have been given any credibility by bank management.
• VAR calculations for illiquid securities are meaningless
• Very large positions can be illiquid because of their size, even though the underlying security is traded in a liquid market.
• Counterparty risk should be stress tested
• Securities that are too illiquid to be exchange traded should have higher capital charges
• Internal risk disclosure by traders should be a key to bonus treatment.  Losses that were disclosed and that are within tolerances should be treated one way and losses from risks that were not disclosed and/or that fall outside of tolerances should be treated much more harshly for bonus calculation purposes.
• Banks did not accurately respond to the Spring 2009 stress tests
• Banks did not accurately self assess their own risk management practices for the SSG report.  Usually gave themselves full credit for things that they had just started or were doing in a formalistic, non-committed manner.
• Most banks are unable or unwilling to state a risk appetite and ADHERE to it.
• Not all risks taken are disclosed to boards.
• For the most part, losses of banks were < Economic Capital
• Banks made no plans for what they would do to recapitalize after a large loss.  Assumed that fresh capital would be readily available if they thought of it at all.  Did not consider that in an extreme situation that results in the losses of magnitude similar to Economic Capital, that capital might not be available at all.
• Prior to Basel reliance on VAR for capital requirements, banks had a multitude of methods and often used more than one to assess risks.  With the advent of Basel specifications of methodology, most banks stopped doing anything other than the required calculation.
• Stress tests were usually at 1 or at most 2 standard deviation scenarios.
• Risk appetites need to be adjusted as markets change and need to reflect the input of various stakeholders.
• Risk management is seen as not needed in good times and gets some of the first budget cuts in tough times.
• After doing Stress tests need to establish a matrix of actions that are things that will be DONE if this stress happens, things to sell, changes in capital, changes in business activities, etc.
• Market consists of three types of risk takers, Innovators, Me Too Followers and Risk Avoiders.  Innovators find good businesses through real trial and error and make good gains from new businesses, Me Too follow innovators, getting less of gains because of slower, gradual adoption of innovations, and risk avoiders are usually into these businesses too late.  All experience losses eventually.  Innovators losses are a small fraction of gains, Me Too losses are a sizable fraction and Risk Avoiders often lose money.  Innovators have all left the banks.  Banks are just the Me Too and Avoiders.
• T-Shirt – In my models, the markets work
• Most of the reform suggestions will have the effect of eliminating alternatives, concentrating risk and risk oversight.  Would be much safer to diversify and allow multiple options.  Two exchanges are better than one, getting rid of all the largest banks will lead to lack of diversity of size.
• Problem with compensation is that (a) pays for trades that have not closed as if they had closed and (b) pay for luck without adjustment for possibility of failure (risk).
• Counter-cyclical capital rules will mean that banks will have much more capital going into the next crisis, so will be able to afford to lose much more.  Why is that good?
• Systemic risk is when market reaches equilibrium at below full production capacity.  (Isn’t that a Depression – Funny how the words change)
• Need to pay attention to who has cash when the crisis happens.  They are the potential white knights.
• Correlations are caused by cross holdings of market participants – Hunts held cattle and silver in 1908′s causing correlations in those otherwise unrelated markets.  Such correlations are totally unpredictable in advance.
• National Institute of Financa proposal for a new body to capture and analyze ALL financial market data to identify interconnectedness and future systemic risks.
• If there is better information about systemic risk, then firms will manage their own systemic risk (Wanna Bet?)
• Proposal to tax firms based on their contribution to gross systemic risk.
• Stress testing should focus on changes to correlations
• Treatment of the GSE Preferred stock holders was the actual start of the panic.  Leahman a week later was actually the second shoe to drop.
• Banks need to include variability of Vol in their VAR models.  Models that allowed Vol to vary were faster to pick up on problems of the financial markets.  (So the stampede starts a few weeks earlier.)
• Models turn on, Brains turn off.

Whose Loss is it?

As we look at the financial system and contemplate what makes sense going forward, it should be important to think through what we plan to do with losses going forward.

There are at least seven possibilities.  As a matter of public policy, we should be discussing where the attachment should be for each layer of losses.  Basel 2 tries to set the attachment for the fourth layer from the bottom, without directly addressing the three layers below.

So for major loss scenarios, we should have a broad idea of how we expect the losses to be distributed.  Recent practices have focused on just a few of these layers, especially the counterparty layer.  The “skin in the game” idea suggests that the counterparties, when they are intermediaries, should have some portion of the losses. Other counterparties are the folks who are taking the risks via securitizations and hedging transactions.

However, we do not seem to be discussing a public policy about the degree to which the first layer, the borrowers, needs to absorb some of the losses.  In all cases, absorbing some of the losses means that that layer really needs to have the capacity to absorb those losses.  Assigning losses to a layer with no resources is not an useful game.  Having resources means having valuable collateral or dependable income that can be used to absorb the loss.  It could also mean having access to credit to pay the loss, though hopefully we have learned that access to credit today is not the same as access to credit when the loss comes due.

+    +    +   +

This picture might be a useful one for risk managers to use as well to clarify things about how losses will be borne that are being taken on by their firm.  The bottom layer does not have to be a borrower, it can also be an insured.

This might be a good way to talk about losses with a board.  Let them know for different frequency/severity pairs who pays what.  This discussion could be a good part of a discussion on Risk Appetite and Risk Limits as well as a discussion of the significance of each different layer to the risk management program of the firm.

The “skin in the game” applies at the corporate level as well.  If you are the reinsurer or another counterparty, you might want to look at this diagram for each of your customers to make sure that they each have enough “skin” where it counts.

Counterparties

When you substitute counterparty risk for another risk, you are essentially bringing their entire balance sheet proportionately onto yours.  Counterparty due diligence is key.  Collateral agreements are important.  Some would say that collateral agreements brought down the banks that failed and AIG that was rescued, but from the counterparty point of view…  In addition to traditional credit analysis that is mostly backward looking, insurers should try to understand the approach to risk taking of their counterparties so that they can become comfortable with the risks that they may take in the future.  The counterparty exposure that exists right now may not be representative of the size of the exposures right after a major loss event.  Examination of those potential exposures and the potential losses to the reinsurer in a major loss event should be studied and factored into risk and reinsurance decisions.

This means plotting the level of obligation from the counterparty in the event of an extremely adverse scenario.  That is when the idea of taking on a proportionate share of the counterparties balance sheet takes on significant importance.  The degree to which the counterparty is concentrated in that particular risk becomes key.  That is not information that is available from just looking at the rating of the counterparty.  You must know and understand the other obligations of the counterparty to know the degree to which they are at risk from the type of event that you are offsetting (not transferring see Bad Labels ).

This means that a stress test becomes most important.  The stress test will look at (1) the amount of gross loss, (2) the amount due from the counterparty under the stress scenario in the form of a claim, a reserve credit, or collateral and (3) the degree to which the stress scenario impacts the ability of the counterparty to make good on their obligations.  As was seen during the financial crisis, the liquidity of the counterparty under stress may well be the constraint.  If your firm does not have the liquidity to easily pay the gross losses under that are due in cash, then you are relying on the counterparty as a source of liquidity.

Models & Manifesto

Have you ever heard anyone say that their car got lost? Or that they got into a massive pile-up because it was a 1-in-200-year event that someone drove on the wrong side of a highway? Probably not.

But statements similar to these have been made many times since mid-2007 by CEOs and risk managers whose firms have lost great sums of money in the financial crisis. And instead of blaming their cars, they blame their risk models. In the 8 February 2009 Financial Times, Goldman Sachs’ CEO Lloyd Blankfein said “many risk models incorrectly assumed that positions could be fully hedged . . . risk models failed to capture the risk inherent in off-balance sheet activities,” clearly placing the blame on the models.

But in reality, it was, for the most part, the modellers, not the models, that failed. A car goes where the driver steers it and a model evaluates the risks it is designed to evaluate and uses the data the model operator feeds into the model. In fact, isn’t it the leadership of these enterprises that are really responsible for not clearly assessing the limitations of these models prior to mass usage for billion-dollar decisions?

But humans, who to varying degrees all have a limit to their capacity to juggle multiple inter-connected streams of information, need models to assist with decision-making at all but the smallest and least complex firms.

These points are all captured in the Financial Modeler’s Manifesto from Paul Wilmott and Emanuel Derman.

But before you use any model you did not build yourself, I suggest that you ask the model builder if they have read the manifesto.

If you do build models, I suggest that you read it before and after each model building project.

The Black Swan Test

Many commentators have suggested that firms need to do stress tests to examine their vulnerability to adverse situations that are not within the data set used to parameterize their risk models. In the article linked below, I suggest the adoption of a terminology to describe stress tests and also a methodology that can be adopted by any risk model user to test and
communicate a test of the stability of model results. This method can be called a Black Swan test. The terminology would be to set one Black Swan equal to the most adverse data point. A one Black Swan stress test would be a test of a repeat of the worst event in the data set. A two Black Swan stress test would
be a test of experience twice as adverse as the worst data point.

So for credit losses for a certain class of bonds, if the historical period worst loss was 2 percent, then a 1BLS stress test would be a 2 percent loss, a 4 percent loss a 2BLS stress test, etc.

Article

Further, the company could state their resiliency in terms of Black Swans. For example:

Tests show that the company can withstand a 3.5BLS stress test for credit and a 4.2BLS for equity risk and a simultaneous 1.7BLS credit and equity stress.

Similar terminology could be used to describe a test of model stability. A 1BLS model stability test would be performed by adding a single additional point to the data used to parameterize the model. So a 1BLS model stability test would involve adding a single data point equal to the worst point in the data set. A 2BLS test would be adding a data point that is twice as bad as the worst point.