Is this just MATH that you do to make yourself feel better?
Megyn Kelly asked that of Karl Rove on Fox TV on election night about his prediction of Ohio voting.
But does most risk analysis fall into this category as well?
How many companies funded the development of economic capital models with the expressed interest in achieving lower capital requirements? How many of those companies encouraged the use of “MATH that you do to make yourself feel better” MTYDTMYFB
Model validation is now one of the hot topics in Risk model land. Why? Is it because modelers stopped checking when they got the answer that was wanted, rather than working at it until they got it right? If the later was the answer, then there would be zero additional work to do to validate a model. That validation work would already be done. MTYDTMYFB
The Use Test is quite a challenge for many. First part of the challenge is to produce an example of a situation where they did modeling of a major risk decision before that decision was finalized. Or are the models only brought into play after all of the decisions are made? MTYDTMYFB
There are many other examples of MTYDTMYFB. Many years ago when computers were relatively new and dot matrix printers were the sign of high tech, it was possible to write a program to print out a table of numbers that had been developed somewhere else. The fact that they appeared on 11 x 14 computer paper from a dot matrix printer gave those numbers a sheen of credibility. Some managers were willing to believe then that computers were infallible.
But in fact, computers, and math, are about as infallible as gravity and about as selective. Gravity will be a big help if you need to get something from a higher place to a lower place. But it will be quite a hindrance if you need to do the opposite. Math and computers are quite good at some things, like analyzing large amounts of data and finding patterns that may or may not really exist.
Math and computers need to be used with judgement, skepticism and experience. Especially when approaching the topic of risk.
Statistics works like gravity helping us take things downhill when you are seeking to estimate the most likely value of some uncertain event. That is because each additional piece of data helps you to hone in one the average of the distribution of possibilities. Your confidence in your prediction of the most likely value should improve.
But when you are looking at risk, you are usually looking for an estimate for extremely unlikely adverse results. The principles of statistics are just like the effect of gravity on moving heavy things uphill. They work against you.
Take correlation, for example. The chart above can be easily reproduced by anyone with a spreadsheet program. RISKVIEWS simply created two columns of random numbers that each contained 1000 numbers. The correlation of these two series for all 1000 numbers is zero to several decimal places. This chart is created by measuring the correlation of subsets of that 1000 that contained 10 values.
What this shows is how easy it is to get any answer that you want. MTYDTMYFB