The recent Wall Street Journal article “Odds-On Imperfection: Monte Carlo Simulation” asserts that Monte Carlo simulation did not predict the market crash, and cites a chorus of critics calling for a fix to the technique. The article equates the technique of Monte Carlo simulation with the models that are using it – two very different things. For instance, the article states, “These models were supposed to help quantify and manage the risks of mortgage-backed securities, credit-default swaps and other complex instruments. But given the events of the past couple of years, it appears that the models often gave big institutions, as well as small investors, a false sense of security.”
This is true – the models for decision making under uncertainty gave a false sense of security. But that’s because the assumptions underlying the risk analysis models were flawed, not because the technique of Monte Carlo simulation was problematic. Monte Carlo simulation is simply a mathematical technique that recalculates many different possible scenarios – but only within boundaries set by the user. You can’t change the underlying math behind these “what-if” calculations.
The article comes close to making this distinction in one sentence: “Critics emphasize that the problem isn’t Monte Carlo itself, but the assumptions that go into it.” It then goes on to describe efforts by firms to include “fatter tail” distributions that more accurately reflect the probability of extreme events occurring as an effort to improve Monte Carlo simulation. Tools like @RISK (risk analysis software add-in for Microsoft Excel) allow complete control over the definition of many dozens of distribution types, enabling users to create as fat a tail as they want. While these efforts make sense, it should be made clear that these are changes to underlying model assumptions, not changes to Monte Carlo simulation itself. To equate Monte Carlo simulation as a technique with the probability distributions people decide to use is to equate a carpenter’s choice of nails with his hammer.
Finally, the article cites the need to run tens or hundreds of thousands of scenarios, instead of just 100 or 1000. This too is user-defined, and tools like @RISK can run as many scenarios as desired.