Computers don’t work when you lie to them

September 22nd 2008 09:16 pm

Here is a terrific (but depressing) article by Saul Hansell explaining how the Wall Street meltdown was fueled by feeding nonsense to the risk management systems in the big investment houses.

The systems did not have models of those weird derivative instruments being traded, so traders would say they were trading a generic (safe, well-understood) loan instrument. So the systems did not really model the risk.

I find this really heartbreaking. I have to believe some people behind the scenes knew what was going on, and I can imagine them losing the argument with their bosses when they tried to fix things.

Risk analyst (RA): Boss, there’s a problem with the risk management system.

Big Boss (BB): I’m sure the system is fine. Look, we’re making tons of money and the system says we’re perfectly safe!

RA: The system doesn’t understand these complex mortgage securities, so it doesn’t accurately reflect the risk.

BB: Do you know how to accurately model those securities?

RA: Well, no, but if we spent some time studying it, we’ll come up with something more accurate than what we have now.

BB: What will the more accurate model show?

RA:No doubt it will show that the securities carry a lot of risk, more than the current models show.

BB: And that will require us to increase our cash reserves?

RA: Yes.

BB: And that will reduce our liquidity, and thus reduce our ability to invest and get a return on our capital?

RA: Yes.

BB: And our competitors, like us, are making good money on these instruments?

RA: Yes, but…

BB: And the regulators aren’t telling us to do this, are they?

RA: No, but…

BB: And you aren’t sure that this new model will be accurate?

RA: Yes, but…

BB: So we lower our profitability as part of debugging some computer model?

… And so on.

A really gutsy risk analyst might bring out this:

RA: The improved model could keep us from digging ourselves into a huge problem with these securities.

BB: So we’re risking the company if we don’t improve our model? You really believe that? The whole industry is trading these things. Someone must have modeled the risk even if we didn’t. …

And the rest is history.

Information Security?

Sorry, we were talking about “securities” and not “security.” This Wall Street meltdown (crash?) is in part due to failures in computerized risk management models. That failure is in part because people were not inclined to build completely accurate computer models. So the inaccurate models didn’t ring any alarms as things got worse and worse.

What most computer based security systems try to do is model some aspect of the real world: the existence of trust relationships between some people and the distrust of others. The systems break down when they have inaccurate information of any sort.

We often face situations when we want to misrepresent the truth. Some are very hard to identify.

The stakes get incredibly high when we’re working with automatic systems.They make their decisions - right or wrong - very quickly and may act automatically. The 1987 crash is blamed on automated buy/sell systems that reacted automatically to a set of bad conditions, which triggered a cascade of sell orders.

Posted under Information Security |

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.