Stochastic Monte Carlo simulations (SMCS) of insurer activities have been used to create nearly continuous distribution curves of expected gains and losses. Economic Capital models are often aggregations of these separate SMCS models to create a similar distribution of total group gains and losses. There are several primary characteristics of the results of these models:
- They produce a nearly infinite number of numerical results giving an incredibly rich vision of the possibilities for the results of the activities of the firm.
- That tidal wave of numeric results is very difficult to digest and make sense of.
- While the modelers may have developed methods for validating the models, it is extremely difficult for general management who are supposed to be the primary users to “validate” the models and therefore, very difficult for them to trust the models
What would validation mean for the general managers? It would mean that they would have confidence that the experiences that they have of the company’s risks and their expectations of future experience would be consistent with the model.
Ultimately, managers should have the same sort of reliance on the model that they have on the speedometer in their car or the clock on their wall. The same sort of confidence that a cook might have on the thermometer on the oven. They get that confidence not by having an expert show them a report, they get that confidence by experience. The speedometer tells them that they are driving within the speed limit and they go past a police speed trap and they are not stopped for speeding. They leave for work with enough time to get there and they arrive on time. The cook puts the cake in the oven and it does not burn and it does cook appropriately in the time expected. Not just once, but over and over again.
The problem is trickier for the SMCS model. The output is really a set of likelihoods. But when that output is presented as the infinite stream of numbers, there is no intuitive way to validate it naturally against experience. And also, when the output is presented as a single remote number, like a 1 in 200 loss, it is also nearly impossible to validate naturally, by experience.
Tranching a security means splitting up the cashflows in some particular, predetermined way. The idea of tranching can be used to help with promoting the natural validation process for a SMCS model. The future possibilities can be tranched into 6 or 8 natural stories. Then the model results can be sorted into the scenarios that match up with the stories. The model output can be characterized as predictions of likelihood for the stories. Here are some possible stories:
- Highly favorable results – Bonuses are maximized
- Favorable results – Bonuses are above expected/average
- Somewhat unfavorable results – Bonuses are paid but below average/expected
- Unfavorable results – no bonuses are paid
- Highly unfavorable results – Layoffs and/or executive firings
- Critical Loss – Company has to drastically change activity – may go into runoff
- Disaster – company insolvent – seised by regulator
Management can participate in defining the range of results that frame each story there. Then the model can provide its prediction of likelihood of each tranche. Management can also provide their prediction of the likelihood of each tranche. The stories do not have to be compensation related. Riskviews has found that if the bonus program of a company was thoughtfully constructed, there are likely to be other stories that can be told of company experience to define the same ranges of results.
A seriously valuable and interesting discussion might result.
Riskviews was once an executive manager for a business unit within an insurer. The insurer’s risk model was used to produce a projection of what might happen to that business in an adverse market. Riskviews response was to ask on which day of that crisis the modeler was predicting that Riskviews was going into a coma, because the business decisions that are predicted would never happen if Riskviews was conscious.
These stories can be used to promote the validation process by the managers. At the conclusion of each period, the modelers and the managers can review the actual experience in terms of the stories. Then they can all decide if the experience validated the model or if it provided experience that suggests recalibrating the model.
Now if this process happens once per year, then it will take a very long time for that natural validation to take place. Probably longer than the tenure of any single management team. And as the management team turns over, the validation process is likely going to need more time. Therefore, it is highly recommended that this process be repeated quarterly. And perhaps repeated for each of the sub models of the economic capital model.
The way that the cook or the driver or the commuter got to rely on their tools was by repeated experiences. The same sort of repeated experience is needed to validate the SMCS model in the minds of the management users.