Monte Carlo Simulation

In my last blog entry I introduced the notion that optimal decision making wasn’t ‘on the radar’ for many clients in Australasia, and laid out a couple of ideas why. I too once focussed on Monte Carlo simulation rather than decision evaluation, but last year the most obscure event changed that.

Call me a nerd of you will, but I like modelling problems in Excel. There is skill involved in setting up a problem such that the model assumptions aren’t too gross, and an art to making the model elegant. This elegance can be very important to optimisation problems, but more on that later. My first homemade optimisation problem was generated by motorcycle racing! MotoGP, to be precise. A friendly tipping competition with friends was formed at the start of the 2009 season with the following structure:

• Entrants played the role of Team Manager.
• Team Managers had a fixed budget to spend on riders.
• Either a few good riders could be purchased, or many lesser riders, or something in between.
• The team that had accumulated the most points at the end of the season was the winner and received kudos!

Although the future results could not be known of course so I set up and ran the optimisation with Evolver after the event to see what the optimal team selection would have been. Historical data could have been used to discover the type of rider mix that tended to be optimal and thus make an informed decision for this competition. The risk in having only a few riders was that any misfortune would have a big negative impact on the points won, whereas a team consisting of many (cheaper) riders was less likely to suffer such a fate. This downside scenario will be modelled into the 2010 MotoGP Team Manager predictive, optimised model (currently in production)!

What has this to do with the corporate world? Replace “team” with portfolio and “riders” with “assets”, “shares” or “projects” and you have a classic portfolio optimisation model. I hadn’t created this model with business applications in mind but I realised that was precisely what I was doing. An instant later I realised just how useful Evolver would be in many decision scenarios even though it doesn’t incorporate uncertainty (RISKOptimizer does).

In the next instalment I will further explore some practical applications for Evolver and you’ll see just how universally appropriate it can be.

» Making Optimal Choices, Part 1

Rishi Prabhakar
Trainer/Consultant

Risk analysis and decision-making tools are relevant to most organisations, in most industries around the world.  This is demonstrated by the speaker line-up at this year's European User Conference, an event at which we believe it is important to bring together customers from a wide range of market sectors.

We are holding 'New Approaches to Risk and Decision Analysis' at the Institute of Directors in central London on 14th and 15th April 2010.  As with previous years, the programme aims to provide everyone attending with practical advice to enhance the decision-making capabilities of their organisation.  Customer presentations, which offer insight into a wide variety of  business applications of risk and decision analysis, include:

• CapGemini: Faldo's folly or Monty's Carlo – The Ryder Cup and Monte Carlo simulation
• DTU Transport: New approaches to transport project assessment; reference scenario forecasting and quantitative risk analysis
• Georg-August University Research: Benefits from weather derivatives in agriculture: a portfolio optimisation using RISKOptimizer
• Graz University of Technology: Calculation of construction costs for building projects – application of the Monte Carlo method
• Halcrow: Risk-based water distribution rehabilitation planning – impact modelling and estimation
• Pricewaterhouse Coopers: PricewaterhouseCoopers and Palisade: an overview
• Noven: Use of Monte Carlo simulations for risk management in pharmaceuticals
• SLR Consulting: Risk sharing in waste management projects - @RISK and sensitivity analysis
• Statoil: Put more science into cost risk analysis
• Unilever: Succeeding in DecisionTools Suite 5 rollout – Unilever's story

We will also look at the recently-launched language versions of @RISK and DecisionTools Suite, which are now available in French, German, Spanish, Portuguese and Japanese.  Software training sessions will provide delegates with practical knowledge to ensure they can optimise their use of the tools and implement business best practise and methodologies.

With over 100 delegates from around the world attending, the event is also a good opportunity to network and knowledge-share with risk professionals from around the world.

» Complete programme schedule, more information on each presentation,
   and registration details

Something has troubled me for some time regarding the choices being made in risk land. I train and work with many clients whom have adopted Monte Carlo simulation techniques (via @RISK for Excel) into the day-to-day running of their businesses. By doing so they (hopefully) now have a good understanding of the exposure they are facing be it in project cost estimation, discounted cash flow analysis or, well, anything really. But this is only one facet of risk and decision assessment, specifically dealing with the descriptive statistical output from a simulation. What of the decision evaluation component? Why aren’t more of my customers analysing the decisions they make, or better yet actually optimising them? I have a few ideas why.

If you’re in business you have to make decisions. Big ones, little ones, yes/no, multiple state and continuous value decisions. Decisions that impact other decisions in simple or complex dependency structures. But are you making the best decisions possible? I’m sure important decisions aren’t being made completely randomly (I hope!) but I see many companies who rely completely upon qualitative techniques for their decision making (experience, gut feel, etc.) which of course means optimality is no more than a hoped for outcome rather than something that is actually being worked towards.
Firstly the decision model must be identified and then quantified, and this can be a difficult task. There is a level of modelling aptitude necessary for effective modelling that goes beyond merely knowing Excel and its functions, and into the construction of logical mathematical descriptions of possibly complicated processes. Relevant decisions need to be identified and the impact of those decisions combined into a formula that can be mathematically optimised. A critical component to all this is the knowledge that spreadsheet models can actually be optimised, and that in cases where Excel’s Solver fails there are Palisade products (Evolver and RISKOptimzer) that can perform optimisations under virtually any circumstance.

I too used to focus on Monte Carlo simulation rather than decision evaluation, and this was mainly a product of the clients I was dealing with almost exclusively when I first worked for Palisade. In my next blog I’ll tell you why that changed and also get a little more into the nuts and bolts of optimisation.

Rishi Prabhakar
Trainer/Consultant

Rumors of this death are greatly exaggerated.  

Why is it that most of the high profile projects managed by the government in the UK all ultimately become beset by problems? A number of projects jump to mind – the Millennium Dome, Wembley Stadium and currently the NHS IT. All three have been plagued by developmental delays and financial mismanagement.

Recently, yet another worthy, but ambitious project has been announced – the North-South high speed rail line to connect London to Scotland. One wonders if the government undertakes detailed quantitative project risk analysis for its infrastructure initiatives?

A good example to highlight in this context is ENGCOMP, a Saskatchewan-based engineering consulting firm that has worked with the Canadian Department of National Defence (DND) to help define budgets for the fourth phase of construction of its Fleet Maintenance Facility at Canadian Forces Base Esquimalt in Victoria, British Columbia. Using @RISK, a Monte Carlo simulation tool, ENGCOMP helped the DND define and secure budget approval from the Federal Government’s Treasury Board. The consultancy firm was able to estimate the impact of the variability and uncertainties pertaining to risks, costs and scheduling. This assessment enabled it to estimate the project risk budget or the risk reserve and schedule contingency, which were both factored in when defining the total project cost of the infrastructure project.

The fact is, in the world of business, risk is inherent and unavoidable. Whilst one cannot completely control risk, one can certainly help reduce uncertainty, greatly increasing the chances of project success. For instance, a key finding of the project risk analysis conducted by ENGCOMP was that, taking into account all the risk and uncertainties on the project, there is an 85 per cent chance that the Fleet Maintenance Facility project will be completed in January 2014. A fairly positive result for the DND, given the scale and complexity of this project in question.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

In the last two blogs I have challenged the idea that simulation results can be boiled down to a single statistic with any positive benefit. The context of a statistic is incredibly important, which is another reason why many statistics and charts/tables should be reported on, not simply one figure. And here’s a compelling reason why.

Consider two competing, similarly-sized projects, of which a company can only pursue one. Now let’s say this company would like to take on the project that has the “least risk”. If they are only familiar with generating the P90 for the total project cost they will be forced to select the project with the lowest P90. But what if the key drivers for exceeding the P90 are easier to mitigate in one project compared to the other? Perhaps the project with the lower P90 also has a higher P95 or P99 – this means the catastrophic failure is actually greater despite a lower P90 and is the mathematical equivalent of “when things go bad, they go really bad”. Not all P90s are created equally! Such an adverse outcome might sink a smaller company where a larger one could wear the loss. The context of the company running the analysis also impacts the context of the analysis itself.

So you can see not only do simulations generate results with which informed decisions can only be made if approached holistically, but if the language used is restrictive this outcome will never be achieved. Risk analyses are a necessary part of business because most of us wish to minimise the chance that something bad will happen, quite simply. Even if a manager tells you they “want the P90” what they are really asking is “tell me about the risk we’re facing”. The answer to this fundamental question is not found in a single figure taken from a simulation, but in a range of charts and tables which require correct interpretation.

More so, Monte Carlo simulation itself is only one piece of the risk and decision assessment pie. Decision modelling and optimisation, predictive modelling and statistical analyses should also form part of the quantitative approach to uncertainty. There is life beyond just risk simulation software, and I intend on exploring that in future blogs.

» Part 1
» Part 2

Rishi Prabhakar
Trainer/Consultant

Where I left off last time was lamenting the use of Monte Carlo simulation to create a single value (statistic etc.) from a model. It might still not be clear why this is anathema to me, so here goes.

A simulation is not a number. It’s not one possible (future) outcome – that’s a scenario. Monte Carlo simulation is a methodology for understanding one’s exposure to outcomes not situated close to the central tendency of the process/project in question. Note the plural “outcomes”. Risk analysis, when done properly, should let you know essentially all possible outcomes and how likely they are for your model. Output from a simulation can include a plot of means (over time), or P5s, or P95s, or the mean ± one standard deviation or any number of statistics. But that’s not plotting a simulation! Let’s not give a minimalist graph too much credit.

Such statements also perpetuate the idea that simulation is only used for creating means (or other centrally tending statistics) and ignores the wealth of information available. Risk simulation software exists to help you do risk analysis which must include not only several statistics but also sensitivity information. It is all too easy to turn a risk assessment into a hunt for a regularly asked for percentile (such as the P90) and there ends the task. I see this a lot, especially in project cost estimation where the pressure both from management and regulatory bodies is to accurately estimate some large percentile. Once found there is usually scant further risk analysis.

Nothing good ensues. When risk analyses are run “to get ‘the’ number” they become simply another box to tick in a process and ultimately any benefits (perceived or actual) will be forgotten and lost to the ages. The notion of context is also lost. No single number by itself really means anything, or at least shouldn’t mean anything to a decision maker. I have often heard phrases like “the model returned/gave $1.2m” followed by an audience nodding in agreement. Huh? Which statistic are you talking about there, and how about reporting a few other numbers around it to place that $1.2m somewhere meaningful?

In the next installment I will look further into this issue of context and hopefully prove the necessity of an holistic approach to understanding and reporting simulation results.

» Part 1

Rishi Prabhakar
Trainer/Consultant

I read an interesting article on the causes of the Global Financial Crisis by John B. Taylor. Although the topic is interesting enough already, especially for a member of a risk analysis-specialising company, something else caught my eye. I have observed in training workshops, onsite consulting and now academic papers a phenomenon regarding probabilistic modelling. Many of those using the methods don’t understand what they should actually be getting from the methodology. There is an intellectual leap from the deterministic to the probabilistic that sometimes does not get made. This limits the usefulness of Monte Carlo simulation, and the value of performing such statistical analyses.

Back to the article which spurred me to write this blog in the first place. Or rather, the graph. Yes a single graph of housing starts vs. time (and its brief description) leapt out at me. One of the lines on the graph was claimed to show model simulations of housing starts using the actual interest rate, compared to the interest rate ‘predicted’ by the Taylor Rule and a third line showing actual data.

So what’s the problem?

The problem is that simulation techniques should not be used to create a single value. The single ‘simulation’ line implies a single modelled/returned value for each time period. This is deterministic modelling. There may be a particular scenario that has been modelled, but it certainly isn’t a simulation that is being represented by that single line. Simulations produce thousands of data, observed values and their associated percentiles as well central moments (mean, variance etc.). Not just one value (sorry Value at Risk – that includes you too) that can be plotted as a single line. I would guess that if a simulation were run as I understand the term then the line in the chart was probably constructed using the simulated means. But I shouldn’t be guessing.

This is far from the only time I’ve seen simulation results reduced to a single entity. I have heard from clients in the past “the simulation gave $X” with little to no context around it, and this is supposed to both mean something to me and to their customers and help to make better decisions under uncertainty…

In the next blog I will explore this idea further and discuss the sorts of results that should be gleaned from a simulation. In particular, why narrowing simulation results down to a single number is counterproductive to healthy business practices.


Rishi Prabhakar
Trainer/Consultant

In Part 2 of this series I finished by asking what should be done with historical data, now that we have decided that storing it is probably a good idea. I won’t keep you waiting any longer.

Auditing and calibration of the model at both the micro and macro level. It’s as important as any other element of risk or statistical analysis, or indeed the model building itself. At the distribution level historical data helps to both parameterise the distributions and in fact select them in the first place. As a minimum a few data points will help you to understand possible central tendencies and variability for your risks, and also generate a list of feasible distributions to choose from. With a reasonable number of observations @RISK for Excel can be used to fit distributions to the data taking care of both distribution selection and parameterisation simultaneously. Only five data points are technically needed, but a reasonable fit will require either more than that or other holistic information to achieve validity.

At the macro level total project cost estimates are often ignored from the portfolio perspective. Commonly high percentiles are reported from such models to use in a ‘contingency’ calculation, such as the P90 or P95. Whilst a high percentile, the P90 (say) should still be exceeded 10% of the time! If your projects never go over this percentile then either there are some major mitigating factors not included in the model or the volatility is being consistently overstated. Likewise, the P10 for total cost (these ‘good’ percentiles are rarely if ever reported or considered in project cost estimation work) should be bettered in roughly 10% of projects. If this is not the case then the upside risk has been overstated. This may be due to misconceptions about the positive skewing present in most cost/delay risks or mistakes made in the parameterisation of the risks where the estimate (“most likely” etc.) is actually the “best case” or close to it, rather than a central tendency of the process over time. There could also be other possibilities.

No matter how you look at it, the collection and intelligent use of historical data is integral to effective and useful risk analysis and management, and critical to achieving valid Monte Carlo simulation results. If you aren’t currently recording everything you can get your hands on start right now!

 

» Part 1
» Part 2

Rishi Prabhakar
Trainer/Consultant

This Thursday, 28 January 2010 at 11am ET, Dr. William Strauss, President of FutureMetrics, will present a free live webcast entitled, "Simulating the U.S. Economy: Where will we be in 100 years?" Sign up now to attend the webcast.

There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

To see where this growth imperative will take us we first have to see how we go to where we are today. This free live webcast first models the 20th century. The model is both complex and simple. The basic schematic of the model’s relationships is easy to understand. Furthermore, the core of the model is a simple production function that combines capital, labor, and the useful work derived from energy to generate the output of the economy. Complexity is contained in the solutions to the internal workings of the model. What is unique is that there are no exogenous economic variables.  Once the equations’ parameters are calibrated, setting the key outputs to “one” in 1900 results in their time paths very closely predicting the U.S. GDP and its key components from 1900 to 2006. 

The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? From 1900 to 2006 there are periods in which there was parameter switching. (The optimal parameters and the years for the switching were found using a constrained optimization technique.) That suggests that in the future there will also be changes. The experiment uses @RISK’s features (risk analysis software using Monte Carlo techniques) to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization using Monte Carlo simulation) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

----
William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics).

» Register now for this FREE live webcast
» View archived webcasts

In my last blog I mentioned a ‘fact’ about data that came up during a recent public training course (Decision-Making and Quantitative Risk Analysis). This fact stuns me every time I think about it, and certainly floored me the first time I encountered it. So many companies just don’t have it.

Data, that is. Historical data from completed projects, sometimes billion-dollar projects, is simply not collected especially in resources and infrastructure cost estimation. Instead every risk is re-estimated from scratch in every new project based entirely upon an estimator’s recollections or guesses. This is not a suggestion that estimators don’t know what they’re talking about, rather that the benefits of adding historical data to the analysis far outweigh the cost of gathering the information in the first place.

I first worked in the banking sector, hence my surprise to learn of this lack of data storage in certain areas of risk analysis. Project cost estimation, especially in resources and infrastructure – I’m talking to you. In financial circles there are literally millions of data points collected daily across the entire organisation. Gathering data (and then analysing it for some benefit) is simply ‘what we do’, and this process isn’t challenged. Some of the data is quite ‘small’, such as the number of seconds a particular caller was kept on hold before being answered, and others are quite ‘big’, such as multi-million dollar losses due to fraudulent activities. Regardless, it’s all kept in the knowledge that information is power – in this case the power to make intelligent decisions in the future.

How can you judge the efficacy of an estimation process (workshops etc.) if you don’t track the final observed outcomes specifically to make such a judgment? Well, you can’t. And that leaves your company’s risk and decision assessment process in limbo. Without measurement there can be no process improvement or corporate learning. Are you ‘passing’ or ‘failing’ with your use of Monte Carlo simulation via risk analysis software?

Generally the observed outcomes for risks in models will be near the estimated value, and this is to be expected. However the main role of risk analysis is to adjudge exposure to the unexpected. Far too many cost estimation models have very little volatility in their line items. I am very curious to know just how often the realised value of a given line item is outside the range of “possible values” as defined in the model. And what about the total project costs overall? This hints at and leads to the big question which is what could/should be done with such data if it were to be recorded?

I shall address these questions in the next blog. I know you’re excited to find out!

Rishi Prabhakar
Trainer/Consultant

Today there is a heightened appetite for risk management due to global economic circumstances. But risk management has always been an intrinsic aspect of business to a higher or lesser degree. However, in the current technology-led business environment, the use of software to effectively manage risk makes logical sense. It provides a level of sophistication that the traditional processes simply cannot offer. Let me explain why.

Risk management essentially involves three stages – identification, quantification, and the on-going management of risks. In reality, these stages are not completely distinct from each other, with each stage influencing and informing the others. For example, an initial quantification of risks may lead to the conclusion that some of the identified risks are in fact not serious enough to warrant further consideration, or that the original description of the risk was not sufficiently precise for meaningful risk management measures to be put in place.

Each of these stages can benefit from the use of supporting risk modeling software. For instance, Microsoft Excel can be used to create a risk register, i.e. a database that records the risks identified, the assessment of the likelihood and impact of each of these risks, the mitigating actions that have been planned, and the assignment of responsibilities for these actions. However, there are many other software tools available, each designed for a specific purpose and focus. To illustrate, enterprise-wide risk management software focuses on the creation of integrated and holistic risk management systems, whereas Monte Carlo simulation and decision tree software place their emphasis on enhancing the quantitative analysis of risks.

The selection of the appropriate risk analysis software should involve very careful thought. The right decision can lead to a very effective implementation, whereas the wrong decision may result in a large amount of wasted investment.

There are some key considerations to bear in mind when selecting the risk modeling software. Choosing software based on how many staff will genuinely be required for the day-to-day risk management process is crucial. It is easy to select software based on the ideal situation that there will be a wide staff involvement in the risk management process. In reality, this may not be possible, potentially resulting in a cumbersome and inflexible solution being chosen over a more stand-alone and flexible application.

Similarly, knowing the level of risk quantification required is important. In fact, best practice risk management now involves the use of quantitative techniques, often using Monte Carlo simulation. When correctly conducted, the process of quantifying risks is rigorous and structured, can expose hidden or biased assumptions, as well as provide a more solid rationale upon which to base the major decisions.

Finally, determining the extent of on-going risk management needed for your business can assist with software selection. 

Needless to say, any software application will be most successful when used by appropriately trained and motivated staff, and when used as a supporting tool within an overall risk management process. Software is not a replacement for process.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Having talked in previous posts as to why it’s important, and today how accessible it is for any size of organisation to adopt a healthy approach to risk, I’ll now take you through my top ten tips on how you can maximize your risk management programme:

1. Get buy-in
Risk management is not an optional extra. It is a business critical tool that is an asset and an integral part of the project. The company culture must be developed to embrace QRM (quantitative risk management) and DMU (decision making under uncertainty) in order that everyone understands their benefits and therefore accepts the need for them.

2. Get budget
Business tools cost money, but managing risk is an investment - not an overhead – and must be regarded as such. Allocating resource and making it a formal business process should be seen as an insurance policy.  Not only will it help organisations make better decisions that will save them money in the long term but, by identifying potential risks and adverse events, it can protect them against unexpected costs in the future.

3. Get words
As with any organisational change, it is essential that everyone is clear on the new processes. Therefore a common risk language – or 'glossary' – needs to be developed to avoid misunderstanding and to ensure a consistent approach to QRM and DMU.

4. Get numbers
Qualitative assessment is essential, but numbers are more powerful – for example the percentage chance of meeting a deadline or budget. Monte Carlo simulation random sampling provides the margin of error for a venture and is a good way to illustrate the consequences of different courses of action. Risk management experts must ensure everyone understands these figures, and accepts them.

5. Get structure
Managing risk in order to make better-informed decisions requires an appropriate organisational structure. Individuals and groups need clearly defined roles, and must then each take responsibility for their own area of expertise.

6. Get lateral
Every organisation has risks that it deals with on a daily basis and which must therefore be factored in to the decision-making model. However, no enterprise operates in isolation, so other external variables must be included. For example, even a small rise in fuel costs could have a major effect on revenues if raw materials need transporting long distances.

7. Get perspective
Political, cultural and social risk factors can be explored by involving all stakeholders.  Investing time and money in consultation and research ensures that businesses have a clear idea of the complete environment in which they operate, and therefore minimise the chances of products and services failing.

8. Get reporting
Risks, and the management of them, must be reviewed regularly – and the programme amended if necessary. This requires a regular reporting process, in which risks are clearly identified and prioritised.

9. Get with it
Being risk aware does not mean being risk averse. Businesses should guard against rigidly adhering to 'the way we've always done it' approach, instead keeping up-to-date, learning new tricks and not being afraid to be bold.  Although risky on the surface, these tactics prevent being left behind – much of the potentially uncertainty can also be removed with QRM and DMU.

And finally…

10. Get it documented
Back up the commitment to a thorough QRM and DMU programme with documentation. This validates the budget and buy-in requested at the start. And it’s good for business – organisations this thorough are guaranteed a competitive edge.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

It’s clear that the financial crisis has exposed a number of failings in the practice of risk management. In my last post I talked about the relevance risk analysis and the disciplines of ‘quantitative risk management’ (QRM) and ‘decision making under uncertainty’ (DMU) are to all sizes of organisations, be it large or small. 

However, how accessible are these disciplines to the average size business across the globe today?

With the need to make more informed decisions more pressing by the day, thankfully QRM and DMU and now far more accessible than ever before.  Traditionally systems tended to be expensive, enterprise-based applications targeted at large companies who were prepared to spend considerable time, money and human resources.  The result was an all-singing, all-dancing product which often ended up underused due to confusion on the part of the very employees who were supposed to make it work.

Steady increases in computer processing have given the desktops of today as much power as the high-end servers of a few years ago, meaning that risk analysis and management is now an achievable goal for businesses of all sizes.  Palisades @RISK and Decision Tools Suite software are such desktop risk and decision analysis tools – working within Microsoft Excel and therefore being accessible to a large number of users.

‘Monte Carlo Simulation’, a technique originally conceived by scientists working to develop the atomic bomb as part of the Manhattan Project, is an inherent part of @RISK, a cornerstone of the Suite.  It enables users to introduce uncertainty into their previously static spreadsheets, which lets them look at things in a probabilistic, rather than a deterministic way.  In layman’s terms, this means that rather than companies and individuals making decisions based on estimates or best guesses, they can see all the potential outcomes to a venture – and how likely these scenarios are to occur.

For many companies this significantly improves the decision-making process.  Firstly it requires a change in the methodology of employees responsible for assessing risks and opportunities and secondly for the first time employees have a tool which allows them to communicate their recommendations to management or colleagues in a transparent and standardised way.  Equally, being able to look at scheduling risk in a probabilistic and quantitative sense allows for the allocation of labour and resources in a way which minimises slack and wastage whilst maximizing ROI.

So, it would seem that the new ‘risk management’ language that is starting to develop in the workplace and being taught to a new generation of managers on MBA courses should be welcomed.  With the accessibility of the technology available to assist them, we need to make sure that organisations do more than just pay lip service to QRM and DMU if they are to reap the rewards.

In my next blog I’ll be giving you the my top ten tips to adopting a health approach to risk, that will help businesses of all sizes maximize their risk management programmes.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

As anyone who does risk analysis is aware, the flip side of opportunity is risk, or maybe opportunity is the upside of risk.  They are always there together, the two sides of chance, but it's great to occasionally see the brighter side of the coin. 

I recently tripped over a very good and interesting article written by Marcia Gulesian, titled Six Sigma, Monte Carlo Simulation and Kaizen for Outsourcing.

Despite its seemingly complex title, the article touches on the basics of Six Sigma and decision analysis where Six Sigma basic quantitative calculations are discussed - such as process capability calculations (Cp, Cpk) are used in an example for the outsourcing of a critical component. The example utilizes a Monte Carlo Simulation a model to illustrate her point.

The example model simulates the outsourcing of a critical component to 3 different vendors, and demonstrates the critical information that a Monte Carlo Simulation model can provide to make informed decisions regarding cost, volumes, supplier capabilities and internal resources. As we all know, having multiple vendors is necessary, but knowing how to distribute your demand across them and knowing the risks and costs involved is critical.

If you would like to experiment with the model, you can download it,  But please know you'll need @RISK to run it, you download the @RISK free trial to run the simulation.

The over-arching topic of the article is that any process can be scrutinized for variation and cost reduction, and in my opinion should be. Companies will continue to outsource more and more so that they may focus on their core competencies. But as this happens, it becomes more imperative that a sound strategy is used to manage the potential outcomes.

I’m very happy to have found this article. Maria obviously understands the power and value of Six Sigma and Monte Carlo Simulation and look foward to future articles from her. 
 

Dr. Robert Ameo is principal of Market Modelers, LLC, with over 20 years’ experience in health care management, marketing and business development. Prior to founding Market Modelers, he served in the corporate development group at Johnson & Johnson. He is a recognized expert and innovator in the modeling and forecasting of new technology adoption and market share. Robert has extensive experience evaluating investment opportunities and their portfolio impact for mergers and acquisitions, venture investing, research development, and marketing efforts. Using his training as a psychologist and his extensive industry experience, he designs and executes targeted market and expert research experiments to quantify the defensible range of possibilities for new technology and product adoption. His forecasts are used both by start-up ventures to create a vision of their potential worth, and well-established biopharmaceutical and medical device companies to understand the true economic (uncertainty adjusted) value of their potential investments. Prior to his industry experience, Robert was VP of Clinical Operations and Utilization Management for a national managed care company. He holds a behavioral science PhD from the University of Miami.

Dr. Ameo will present a case study next week at the 2009 Palisade Conference: Risk Analysis, Applications, & Training,  21 - 22 October at the Hyatt Regency in Jersey City (10 minutes by PATH from Manhattan's Financial District).

See the abstract for his case study below, and see the full schedule for the Conference here.

Interpretive and Ethical Issues in using Monte Carlo Simulations to Support Executive Decision-Making: How to avoid giving your boss impressive, but misleading guidance

Simulations are proliferating throughout the business community powered by a troop of freshly minted MBAs armed with their requisite course on decision sciences and their student versions of @RISK.

Finance organizations are asking their analysts to “do a Monte Carlo.”  Dutifully, the analysts select a handful of “key” variables, assign triangular or Pert distributions, set iterations to 1000, push the simulate button. The laptop’s screen displays a colorful histogram and a sensitivity analysis to add to the PowerPoint.

Lo and behold, the simulation analysis supports the original scenario model showing the mean or median simulated output to be just about in the middle of the distribution. Mission accomplished. Senior leadership is assured that the model has been tested by simulating 1000 potential outcomes. Management moves forward in their pre-decided direction with confidence bolstered by a state of the art Monte Carlo analysis.

This scenario happens every day and for so many reasons it is very wrong.

Using simulations to support executive decision-making introduces ethical concerns that are not present in “most likely case” scenario modeling. In this presentation, Bob Ameo discusses the ethical responsibilities of using simulation models to inform executive decision-making. Specific recommendations are made how to appropriately conduct and present outcomes from simulation models.

Next Week: October 21-22 in NYC

Building on the success of last year’s record-breaking event, the conference will offer a wide range of software training, model building, and real-world case study sessions. Last year, the event drew over 150 practitioners and decision-makers from a broad spectrum of industries. The @RISK and DecisionTools software tracks were more popular than ever. This year, we’re expanding software training with sessions that let you walk through examples and try the tools directly. This will enable you to take some new tips back to the office. Please join us in October for a great opportunity to learn and connect with colleagues.

When performing a cost risk analysis study, one of the key results is the amount of extra monetary resources that is to be added to the project cost baseline to guarantee that the budget is not exceeded at a certain confidence level. Good project risk management strategies must take this into account.

After defining the uncertain variables and risk events that affect the cost performance of the project, we can run a Monte Carlo simulation with @RISK to find out what the range of the total project cost is.  Simulation results can help us to explain the risk exposure that we have in the total cost of the project. The most popular statistics are the mean (average cost), the most likely cost, and the 10th and 90th percentiles.




To determine the contingency to be allocated to the project, we need to define what confidence level we would like to achieve: The higher the contingency level, the larger amount of contingency needed. For example, in the figure above, we are reporting the total cost of the project. Here we can observe that we are showing the 85th percentile that corresponds to a total cost of $7.8M (right delimiter).  We can say that there is only a 15% chance that we will exceed $7.8M, or alternatively, we have an 85% chance that the total cost will be less than or equal to $7.8M.  In the same figure we can also see that the 90th percentile of the total project cost is $8.02M.  We can say then that in order to increase our confidence level from 85% to 90%, we will need to add $220,000 to the total cost.

The calculation of the contingency is then accomplished by using the base cost estimate (BE) before the risk analysis was implemented, and the expected cost (EC) of the simulated results.

Some practitioners separate the contingency into two components: engineering allowance, and management contingency.

Engineering allowance (EA) is the difference between the expected cost and the base estimate:

EA = EC – BE

Management contingency (MC) is calculated using the difference between the cost at certain confidence lever (Cp) and the base estimate:

MC = Cp – EC

In our example, our BE = $6.5M; therefore, engineering allowance EA = EC – BE = 6.86M – 6.5M = $0.36M. 

For the calculation of management contingency, we use a confidence level of 85% so Cp(85%) = $7.8M; therefore, MC = Cp – EC = 7.80M – 6.86M = $0.94M.

In many situations, the suggested contingency might be excessive, so the need for a mitigation study is necessary. We can use the sensitivity analysis tool in @RISK to detect the key drivers affecting our total cost. This is valuable information so that we can concentrate our efforts in reducing the impact of risk events and uncertainties to the total cost. Below, we see a tornado graph with the most important drivers. The analyst will then explore the appropriate mitigation strategies and assess their implementation cost. A second simulation can be run to assess the effectiveness of the proposed mitigation plan, and compare the pre-mitigated and post-mitigated cost distributions.




In following blog posts, I will explain how to distribute the assessed contingency to cost elements and identified risk events in project risk management models.

Javier Ordóñez, Ph.D
Director of Custom Solutions

 

What I love about all this is that apparently, Liquid Stone isn't using risk analysis to get the messy areas better organized,the purpose of it is to figure out how to fit new stuff into the mess.