In the last two blogs I have challenged the idea that simulation results can be boiled down to a single statistic with any positive benefit. The context of a statistic is incredibly important, which is another reason why many statistics and charts/tables should be reported on, not simply one figure. And here’s a compelling reason why.
Consider two competing, similarly-sized projects, of which a company can only pursue one. Now let’s say this company would like to take on the project that has the “least risk”. If they are only familiar with generating the P90 for the total project cost they will be forced to select the project with the lowest P90. But what if the key drivers for exceeding the P90 are easier to mitigate in one project compared to the other? Perhaps the project with the lower P90 also has a higher P95 or P99 – this means the catastrophic failure is actually greater despite a lower P90 and is the mathematical equivalent of “when things go bad, they go really bad”. Not all P90s are created equally! Such an adverse outcome might sink a smaller company where a larger one could wear the loss. The context of the company running the analysis also impacts the context of the analysis itself.
So you can see not only do simulations generate results with which informed decisions can only be made if approached holistically, but if the language used is restrictive this outcome will never be achieved. Risk analyses are a necessary part of business because most of us wish to minimise the chance that something bad will happen, quite simply. Even if a manager tells you they “want the P90” what they are really asking is “tell me about the risk we’re facing”. The answer to this fundamental question is not found in a single figure taken from a simulation, but in a range of charts and tables which require correct interpretation.
More so, Monte Carlo simulation itself is only one piece of the risk and decision assessment pie. Decision modelling and optimisation, predictive modelling and statistical analyses should also form part of the quantitative approach to uncertainty. There is life beyond just risk simulation software, and I intend on exploring that in future blogs.