Reserve Estimation

When doing risk analysis simulation modeling, it’s critical to represent the fact that many variables are related. Very rarely are all variables completely independent of one another.  For example, when interest rates are low, housing starts tend to go up. This relationship is represented by a correlation coefficient, a number between -1 and 1. -1 means the two variables are perfectly negatively correlated;  that is, when interest rates go up 1 unit, housing starts go down 1 unit. A coefficient of 1 is exactly the opposite, and 0 means there is no relationship between the variables at all. Typical coefficients are between these extremes. This correlation between input variables can and must be captured in your financial risk analysis models. Even if you are using expert opinion, estimating these relationships is better than ignoring them completely.

@RISK provides an easy way to define correlation coefficients. Through the use of a slider bar, you can dynamically update coefficients between variables as well as scatter plots representing those variables. This helps you visualize and more easily define these relationships.

For a quick video demonstrating this, see:



» View short videos on recently added @RISK risk software features

On Thursday, September 2, 2010, Svetlana A. Sigalova will present a free live webcast entitled. "The Use of the DecisionTools Suite in Biotechnology Project and Portfolio Decision Making "

Given the uncertainty of outcomes in the biotech industry, consideration of variability is an inherent part of the decision process. Often, the mean (average) is not a relevant decision criteria. This is especially true for smaller biotech companies like Vertex – the opportunity costs are extremely high because scarce capital resources would be invested elsewhere, with a higher probability of realistic return. For example, a company may reject a project which is profitable on average (positive Net Present Value) because some of the possible outcomes are unacceptable to the decision maker. Consideration of variability allows a decision maker to bring in their own risk tolerance into the decision. A similar argument applies when estimating a safety margin above a base case (e.g. in cost budgeting).

» Register now (FREE)
» View archived webcasts

Vertex Pharmaceuticals, Inc. is a global biotechnology company based out of Cambridge, MA. The Company's strategy is to commercialize its products both independently and in collaboration with major pharmaceutical companies. Vertex's product pipeline is focused on viral diseases, cystic fibrosis, inflammation, autoimmune diseases, cancer, and pain.

Given the uncertainty of outcomes in the biotech industry, consideration of variability is an inherent part of the decision process. Often, the mean (average) is not a relevant decision criteria. This is especially true for smaller biotech companies like Vertex – the opportunity costs are extremely high because scarce capital resources would be invested elsewhere, with a higher probability of realistic return. For example, a company may reject a project which is profitable on average (positive Net Present Value) because some of the possible outcomes are unacceptable to the decision maker. Consideration of variability allows a decision maker to bring in their own risk tolerance into the decision. A similar argument applies when estimating a safety margin above a base case (e.g. in cost budgeting).

Vertex’s strategy and analytics group within the corporate finance division seeks to provide the senior management with dynamic revenue and profit forecasting methodology that helps to identify types of drugs that should be developed given a finite amount of cash and resources. A traditional financial view allows the user to identify scenarios and potential outcomes, but lacks the ability to show the range of potential values within each and every outcome. Vertex’s team uses the DecisonTools Suite to establish the average outcome, the variability of outcomes and to pressure-test risk and uncertainty of a particular scenario throughout the decision process.

Vertex’s team built a complex financial risk analysis model using @RISK to enhance its portfolio process. Monte Carlo simulation and optimization are used to analyze and optimize project and portfolio decisions, given short and long-term corporate strategy. @RISK is also frequently used throughout the business development process: simulating across multiple sales forecasts provides BD team with a range of potential outcomes, making it easy to pinpoint a particular scenario on a curve, along with its probability and value. TopRank turns the sensitivity analysis into a quick and seamless exercise, answering multiple what-if questions within minutes. Franchise and program leaders can now see a dollar effect of their program being delayed or advanced, adding supplementary indications to the development plan and even addressing the price uncertainties all at the same time. The simple interface of PrecisionTree along with tornado chart outputs makes it easy to explain the effect and importance of a particular assumption / decision to an audience with no finance background.

As the company continues to grow, adding more drugs and collaborations to its development pipeline, we will see in this free live webcast how the DecisionsTools Suite remains one of Vertex’s analytical tools of choice to enhance and guide the decision making process.

» Register now (FREE)
» View archived webcasts

The UK's new coalition government has said that, as part of its 'Green Deal', it will encourage home energy efficiency improvements paid for by savings from energy bills. It seems likely that, in the year that energy regulator Ofgem warned of 20 percent electricity price hikes by 2020, this initiative will include solar panel technology

Currently the UK still lags behind many other countries in Europe and the rest of the world when it comes to harnessing solar power. Not only do we have less hours of sunshine than many regions, but there is a lack of clarity as to the 'payback' time when it comes to users seeing a return on investment.

This is where Palisade customer, the California-based Tioga Energy, makes an interesting case study. Whilst it may seem unfair to compare the UK with the west coast of America when talking about solar-related matters, the sunnier climate does not reduce the need to prove ROI for customers with solar energy agreements.

Tioga Energy provides project financing through its solar Power Purchase Agreements (PPAs), and maintains and operates solar systems on behalf of its customers. Tioga’s offering delivers predictably priced power and enables organisations to to both 'green' their operations and reduce energy costs. To illustrate the benefits of solar, estimating future electricity prices and making comparisons by showing the savings from a new solar system, Tioga enlisted the help of @RISK for risk analysis solutions.

To forecast possible price increases, Tioga Energy inputs California's historical electricity rate data into a quantitative risk analysis model developed using @RISK. This generates a probability distribution for electricity rate rises over the 20-year PPA period, which shows that there is a 25 percent likelihood that price increases will be less than 4.8 percent, and a 25 percent chance that rate rises would be more than 8.7 percent.

The @RISK risk analysis model therefore helps Tioga Energy evaluate the likelihood that a customer will save money for a variety of PPA scenarios (i.e. the rate at which electricity would initially be charged and the amount by which it would then increase each year). It also calculates the magnitude of savings for the different combinations of first year costs and subsequent rises. Consumers are therefore able to better understand the pricing and make an informed decision about whether to sign up for a PPA.

Using historical data and @RISK's risk modelling software capacity, Tioga offers consumers a robust view of the potential benefits of a solar PPA. This enables them to hedge against rising electricity rates, as well as feel confident that they are playing a part in tackling global warming.

» Read the Tioga Energy case study

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Most people understand that @RISK and Monte Carlo simulation are designed to be an improvement on single-point estimates.  In practice, however, I often see people using @RISK as a forecasting tool to get yet another single-point estimate, such as the 90th percentile, without putting it into the context of the potential range of outcomes.

This is probably the difference between a forecasting and a decision-making.  The former tends to focus on historical or observed trends and developing specific scenarios (e.g. best, most likely, worse) based on expert opinion, while the latter is concerned with confidence ranges and likelihood.

Indeed, it’s not until you add probability, as with @RISK, that you start to measure the quality of your forecasts (i.e. your confidence level) and calculate the margin of error – something that’s crucial in all walks of life!

In my opinion, therefore, @RISK is much more of a decision-making tool than a forecasting tool.  Both involve trying to predict the future but the addition of probability gives decision-makers vital insight to a problem. 

Don’t you just love semantics!

Ian Wallace, ACMA
Palisade Training Team

@RISK, Palisade's risk analysis software using Monte Carlo simulation, has many features that help you create powerful models for decision making under uncertainty.

This model demonstrates the use of the alternate, percentile parameter formulation. In this case we assume that we have decided to use a Normal distribution to represent the arrival time of someone at work. The use of traditional parameters would require knowledge of the standard deviation of the arrival time, which may be hard to estimate. The use of the alternative parameter formulation allows data to be estimated by others in a more natural way. In the first case, the traditional parameters are used (mean and standard deviation). In the second case, the mean is still used, and the P90 is used in place of the standard deviation, i.e. the time before which the person arrives in 90% of cases. In the second case, the P10 and the P90 is used in place of the standard deviation i.e. the time before which the person arrives in 10% of cases, and in 90% of cases respectively.

» Download the example: AltPars.ArrivalTime.xls

One of the side-effects of the recession appeared to be a reduction in the demand for electricity as businesses and consumers alike looked to make savings on their outgoings. However, economic recovery seems to render this trend as temporary, meaning that the global need to tackle energy-consumption is as pressing as ever.

BC Hydro, Canada's third largest electrical utility provides an interesting case study in how to  ascertain the most effective ways to tackle the gap between supply and demand of electricity in British Columbia. Trends such as an expanding population growth, and the increase in energy usage per customer, have led to a rise in the demand for electricity across the region. By legislation, BC Hydro must aim to meet these energy needs through implementing cost-effective energy conservation approaches before it can turn to increasing the supply. 

The company has set itself one of the most aggressive targets in North America, with a plan to meet almost 75 percent of its incremental load through Demand Side Management (DSM) over the next 20 years. DSM projects include compact fluorescent light promotions; subsidies for energy efficient appliances; variable speed motor promotions (for home furnaces); and promotional activity aimed at motivating customers to use less energy.

BC Hydro uses @RISK risk analysis software to measure the uncertainty around its energy conservation efforts, both at the project stage and at a higher portfolio level. Around 60 projects were analysed on a case-by-case basis, and a probability distribution around the forecast outcome was developed. @RISK helps BC Hydro to capture the level of uncertainty of the estimated savings for each individual DSM venture.

In recognition that projects do not operate in isolation, BC Hydro also uses @RISK to explore the interrelationships between key uncertainties: the participation and savings per participant, and the participation across projects. The analysis showed that if a 'conservation culture' was developed in the province, it would result in an increase in energy savings across all programmes. However, it also illustrated that, if this culture failed to materialise, the performance of all programmes will be dragged down.

Exploring uncertainty using @RISK allowed BC Hydro to find the best balance between the uncertainties of supply side resources and those of relying heavily on energy conservation. Employing Decision Making Under Uncertainty helps BC Hydro to meet both financial risk analysis and environmental risk analysis goals. As a result, it expects to meet the majority of its incremental load growth through conservation measures.

» Read the BC Hydro case study

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

"Everything has to run on the principle of profit.  We'll never let creative decisions rule our business decisions.  If it doesn't fit the model, it doesn't get done."  That doesn't mean, he has explained, that if he really likes a project, he and Wilson can't juggle the variables to make the film project fit the model.  They change the parameters to reveal the path to profit.  And profit he has--the estimated assets of Relativity are about $2 billion.  

So Kavanaugh qualifies as a mogul, a math-for-movies mogul.  When the spotlight falls on him, Monte Carlo simulation isn't far out of it.

  

If you missed Palisade trainer Rishi Prabhakar's webcast "Customised Solutions Using @RISK and VBA for Excel," you can still view it in our archive.

The hour-long presentation explores the use of VBA for Microsoft Excel to control @RISK functionality, to simplify the process of risk analysis for resource-strapped businesses. Rishi explains the advantages (and limitations) of macro control for modelling and running simulations.

Simple examples are worked through to show the XDK (@RISK’s automation library) in action, from generic examples to a cost estimation model. This addresses elements of model construction, various simulation settings and finally reporting. The emphasis is on exposing the viewer to the various possibilities the XDK lends to the user rather than an in-depth VBA for Excel coding session.

Rishi Prabhakar holds a BSc in Mathematics from the University of Technology, Sydney Australia. Rishi has experience in the resources, infrastructure and primary industries, telecommunications, scientific research, banking and finance with an emphasis on operational risk.

With technical skills in the areas of modelling, simulation, statistical analysis, cost estimation, time series forecasting, customised solutions utilising VBA for Excel, and extreme value theory, Rishi has provided training and consulting services in risk and decision analysis for Palisade’s Asia Pacific office since 2005.


» Customised Solutions Using @RISK and VBA for Excel
» Webcast archive

Modeling Uncertain Number of Events, Each with Uncertain Parameters
@RISK (risk analysis software using Monte Carlo simulation) is widely used in insurance and reinsurance for premium pricing and loss reserves modeling. A 2006 survey identified @RISK as the third most widely-used software by actuaries, after Microsoft Office and in-house actuarial tools.

@RISK's RiskCompound function allows for the sampling of frequency-severity distributions. This is often required in insurance modelling, as well as in some operations management situations. For example, to determine the total insurance claims payout, one must account for the uncertainty in both the total number of claims (frequency) and the dollar amount of each claim made (severity). 

A powerful feature of the function is that the argument that corresponds to the severity may be a reference to a cell containing a formula (rather than just a single distribution function). 

For example, one could use the function in the form RiskCompound(RiskPoisson(5), A10). The Poisson distribution would describe the frequency (occurrence) of events (e.g. an individual sample may determine that three events occur), and cell A10 would contain a formula that is separately evaluated for each of these three events (before returning the sum of these three as the sampled value of RiskCompound). 

A simple example could be A10 = RiskLognorm(10000,1000)/(1.1^RiskWeibull(2,1)), with the Weibull distribution representing the time to settlement of an insurance claim, which is used to discount the basic claim value sampled from the Lognormal distribution of severity. For example, once a claim is filed for a nominal amount, the actual payment may be delayed due to court actions or disputes, which may reduce the cost of the claim to the insurer.

» Download the model: RiskCompoundCellReferencing.xls

Guest blogger David Roy Six Sigma Professionals, Inc., and taught Jack Welch and his entire staff their Six Sigma Green Belt training. Dave also has a quick survey for your input on structuring DFSS training. brings us the second installment of his four-part blog. Dave comes to us from SSPI,

 

--Steve Hunt

 
Oops! Didn’t see that coming! Part 2

We’d like to ask for your guidance by completing a short marketing survey to help SSPI structure our training in a way that is most useful to our community. This 8 question survey should take less than 5 minutes, and is anonymous. Your opinions are greatly appreciated.

As a continuation from the May blog, we are now covering the “Identify” phase of the ICOV framework of a rigorous new design process.

This phase is important because it establishes the framework for the concept, establishes the level of rigor required for the project management process, estimates the development cost, collects the Customer and Business requirements and the criteria for success.

 

The level of project management needs to be flexible and scalable depending on the Level of Effort (cost) and the Level of Innovation (risk) of the new concept.

 

Surely a project that will take a month to develop and has been done elsewhere requires less rigor that a concept that will take 3 years to develop and represents a brand new invention which has never been done before.

 

The I phase consists of two Tollgates during which an objective steering committee will decide whether to refine the work in the current phase, proceed or cancel the project. 

 

Tollgate 1 Exit Criteria are:

o     Decision To Collect The Voice Of The Customer To Define Customer Needs, Wants And Delights

o     Verification adequate funding is available to define Customer Needs

o     Identification of the Tollgate Keepers1 leader & the appropriate staff

 

Tollgate 2 Exit Criteria is successful demonstration of:

o     Assessment of market opportunity

o     Command a reasonable price or be affordable

o     Commitment to development of the Conceptual Designs

o     Verification adequate funding is available to develop the Conceptual Design

o     Identification of the Gate Keepers leader (gate approver) & the appropriate staff

o     Continue flow down of CTSs to Functional Requirements

 

Formal tools which can be used in this phase are Market/Customer research tools, Product Roadmaps, Process Roadmaps, Technology Roadmaps, Multigenerational plans, Quality Functional Deployment (House of Quality).

 

Market/Customer research tools may include Customer Relationship Management (CRM) Data, Surveys, Focus Groups, Conjoint Analysis and Kano Model Analysis.

 

The next blog will cover the Conceptualize phase

 

 

 

BIO:

 

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. Dave’s experience includes Product and Transactional so his examples are of interest to all. David holds an BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

» Part 1

As a small--very small--shareholder, discovering that there is such a tipping point and that Karame knows how to locate it is reassuring.  Makes me feel there's someone on my side.   

At the 2010 Palisade Risk Conference in London, John Zhao of Statoil used a mock cost estimate contingency model to demonstrate how @RISK simulation functions can yield a more realistic project contingency through integrated qualitative risk assessment and quantitative risk analysis.

While future oil prices may be hard to predict due to low manageability, it is absolutely possible to scientifically forecast the sizes of risks that companies are willing to take, and such risks may include the probabilistic volumes of newly discovered reserves, the probability of meeting a project development schedule, chances of project cost overruns, and the likelihood of eroding entire project profitability. To achieve these goals, @RISK has lent a helping hand to business analysts for easier operation of complicated mathematical modelling.

Statoil, an international oil company, takes risk management seriously and has applied Monte Carlo simulation techniques in core and support businesses using @RISK. Such applications not only include the solo use of individual applications, but integrated combinations from drilling, reserve estimation, and well completion to cost and schedule controls at project execution. Besides the widespread uses of the software, Zhao discussed a specific application of @RISK to convincingly simulate required capital project contingency  in detail.

A simplistic line-item ranging exercise using @RISK Monte Carlo simulation is no longer adequate to derive large capital project contingency, as empirical data confirmed that many disastrous cost overrunning projects were lack of contingency to cover the covert risks. In order to show management a complete risk picture on a project, both systemic risks (which empirical history has indicated a likelihood of occurring), and specific risks (which have discrete probabilistic characteristics), should be included in the overall project risk analysis. Therefore the combination of continuous PDF for project cost estimates, and discrete PDF for project risk registers, may prevail and provide management with a more convincing project cost contingency.

John Zhao is Quality and Risk Manager at StatoilHydro Canada Limited. He has 22 years project management experience in the petrochemical industry. He has authored many papers and made numerous presentations worldwide on the subject of risk and contingency management. In the past 10 years, John has developed his expertise in cost engineering and risk analysis for large downstream and oilsands upstream projects across Canada. His extensive knowledge in construction project qualitative risk assessment process has made him an expert on the subject in North America; his proprietary Monte Carlo model using @RISK is a popular tool for project contingency and escalation simulation. The quantitative model that John has built has integrated @RISK with PrecisionTree to help corporations conduct risk-based strategic decision-making.

» View the complete abstract and PDF presentation of "Put More Science into Cost Risk Analysis"
» Read Zhao's whitepaper, "Put More Science into Quantitative Risk Analysis"

In my last blog I introduced the idea of a customised risk analysis solution to problems commonly faced in project risk management, especially cost estimation. Of course this idea is not uniquely applicable to project costs, but this paradigm is the simplest to explore, and that’s what I’m about to do.

Picture a risk register in a worksheet that has been created at a macro level to encapsulate most (all?) of the risks your projects may face. For any given project only a subset of these will be relevant – what is the best way to get these risks into a risk model on the next worksheet? By pressing a button of course! It is almost trivial to write code that picks up all selected risks and places them and the relevant data fields in the model worksheet. Sure beats manually copying and pasting individual line items and the transcribing errors that follow.

The next problem is utilising the workshopped parameters (likelihood of event, three-point estimates for severity etc.) in a logical way to be referenced by appropriate @RISK functions. Once a model structure has been agreed upon a macro button can place @RISK distributions where they ought to go, either logically due to the paradigm (using RiskBinomial, for example) or via a drop-down selection for dollar impact (RiskPert or RiskGamma, say). My clients have been especially thankful when I limit their choice of distribution and provide a simple flow-chart to follow to make this very decision. Reducing the propensity for arguments in risk workshops is worth its weight in gold; if we can assume that reducing this risk ‘weighs’ plenty!

Similarly one or two instances of the simulation settings are likely to satisfy all requirements, so these too can be activated by macro buttons. In this way a user can’t run a ‘poor’ simulation thus creating spurious results. The simulation output that is required can be placed into a report template attached to the model template and generated using yet another simply-labelled macro button. In this way there will be consistent reporting across the organisation allowing decision makers to become familiar and comfortable with simulation results they might otherwise ignore or be unaware of.

A risk model created by this process may not be the theoretically optimal one, but it will be valid and in context with its intended use. It will certainly be easy to use! The results will be consistent and should satisfy management’s desires as well as regulatory requirements.
The project cost estimation is but one example, and the above possibilities are far from the only ones imaginable. Additional complexity or alternate needs would be just as easily met simply with different code essentially without any practical limits. You don’t need to be an expert in Monte Carlo techniques and software to run robust, credible risk analyses. All you need is a risk analysis consultant who macro-controls the cumbersome and probabilistic elements, some appropriate simulation options and reporting procedures. Ask for me by name!

» Robust Risk Analysis for the Time/Expertise Poor – Part 1

Rishi Prabhakar
Trainer/Consultant

I have recently spoken to several clients whom have all came to the same conclusion about the risk analysis solution they think is most appropriate. They don’t want to do it, and I have no problem with that!

Of course that’s not precisely true. The benefits of Monte Carlo techniques in risk analysis are quite well understood and there is plenty of buy-in from businesses in the Australasian region. The trouble these businesses face (particularly in the realm of project cost estimation) is that the specific process of quantifying their risks for stochastic analysis and the ensuing simulation is not well understood and the means to ameliorate this appears to be beyond their reach. The modelling and simulation components of the project risk management process are not given adequate resources to be performed well, and certainly not to the extent that they provide the most useful information.

It is the case that many companies do not employ dedicated quantitative analysts. This means they have to rely upon some (maybe one) person in the team who has a non-zero quantity of experience and possibly training with risk simulation software to create a valid and credible stochastic model. This person is also not likely to be given enough time to do said task, thus the model inevitably suffers. It is my experience that most models – and all project cost estimation models – can be improved or actually need to be fixed.

So the corporate mind is willing, but the flesh is weak. How can this be addressed? No amount of additional training will suddenly allow you to overcome your time and resource constraints. Perhaps you can’t get the budget for training anyway or don’t want to master risk analysis software when it’s not really core to your role? The solution is one that I personally endorse (and provide!) as a risk analysis consultant – custom Excel programming.

VBA for Excel is a fairly simple language to learn, yet very powerful tool for automating repetitive or sometimes complex spreadsheet tasks. A customised solution involves writing VBA code to perform the tasks we’d rather not do ourselves in the risk analysis model. The “we” here refers to companies that find themselves in the situations previously described whereby they are incapable of creating and operating these models, not necessarily though any fault of their own. In my next blog I’ll examine some modelling problems/requirements and how they might be dealt with effectively using customisation.

Rishi Prabhakar
Trainer/Consultant

Part 2 of this blog ended with me very quickly stating that the MotoGP tipping comp optimiser was identical in structure to a portfolio optimisation problem, where the portfolio could contain stock or other assets, or even projects. Let’s look at this in a little more detail as I’m sure you’re reading this to find how to optimise your own decisions rather than wondering how I went in the tipping competition!

In my model there was a fixed budget (though less could be spent if desired) to spend on riders, with the aim of maximising their total points haul. In the real world you may have a total budget of say $100m to invest in a range of projects perhaps many hundreds of millions of dollars in total value each of which have certain expected returns. At its simplest this decision evaluation will find the most (expected) profitable portfolio of the projects. This is an inclusion/exclusion grouping model, but it is very simple to optimise assets with a continuous level e.g. the amount of money invested in various shares etc. Another real example I have seen when working with an investment company here in Australia was a model whose goal was simply to find the portfolio mix that came closest to the total allowable spend without exceeding it.

Further realism can be included by using constraints should there be the need. A resource constraint may mean there has to be a limit to the number of projects that can be run simultaneously. There may also be a minimum number of projects determined by management as a mitigation strategy. Such constraints are very simple to employ using Evolver and add value to the decision analysis without the need to provide specific risk analysis/Monte Carlo simulation information for the model.

A slightly more sophisticated method of turning an optimisation into a useful portfolio risk management tool where uncertainty hasn’t been specifically modelled is to estimate the possible downside of each asset and include it in the calculation of the portfolio’s ‘score’. The Evolver software comes standard with over twenty example spreadsheets for your educational pleasure, of which “Portfolio Mix.xls” gives one method for doing just this.
In the next (and final) instalment of the Making Optimal Choices blog I will explore the idea that not all optimisations no matter how mathematically correct will produce the same results in good time, and that elegant modelling should always be the goal prior to firing up Evolver.

And so you know, I came second in the competition. Next year I’m hoping to go one better!

» Making Optimal Choices, Part 1
» Making Optimal Choices, Part 2

Rishi Prabhakar
Trainer/Consultant

Risk analysis and decision-making tools are relevant to most organisations, in most industries around the world.  This is demonstrated by the speaker line-up at this year's European User Conference, an event at which we believe it is important to bring together customers from a wide range of market sectors.

We are holding 'New Approaches to Risk and Decision Analysis' at the Institute of Directors in central London on 14th and 15th April 2010.  As with previous years, the programme aims to provide everyone attending with practical advice to enhance the decision-making capabilities of their organisation.  Customer presentations, which offer insight into a wide variety of  business applications of risk and decision analysis, include:

• CapGemini: Faldo's folly or Monty's Carlo – The Ryder Cup and Monte Carlo simulation
• DTU Transport: New approaches to transport project assessment; reference scenario forecasting and quantitative risk analysis
• Georg-August University Research: Benefits from weather derivatives in agriculture: a portfolio optimisation using RISKOptimizer
• Graz University of Technology: Calculation of construction costs for building projects – application of the Monte Carlo method
• Halcrow: Risk-based water distribution rehabilitation planning – impact modelling and estimation
• Pricewaterhouse Coopers: PricewaterhouseCoopers and Palisade: an overview
• Noven: Use of Monte Carlo simulations for risk management in pharmaceuticals
• SLR Consulting: Risk sharing in waste management projects - @RISK and sensitivity analysis
• Statoil: Put more science into cost risk analysis
• Unilever: Succeeding in DecisionTools Suite 5 rollout – Unilever's story

We will also look at the recently-launched language versions of @RISK and DecisionTools Suite, which are now available in French, German, Spanish, Portuguese and Japanese.  Software training sessions will provide delegates with practical knowledge to ensure they can optimise their use of the tools and implement business best practise and methodologies.

With over 100 delegates from around the world attending, the event is also a good opportunity to network and knowledge-share with risk professionals from around the world.

» Complete programme schedule, more information on each presentation,
   and registration details

Something has troubled me for some time regarding the choices being made in risk land. I train and work with many clients whom have adopted Monte Carlo simulation techniques (via @RISK for Excel) into the day-to-day running of their businesses. By doing so they (hopefully) now have a good understanding of the exposure they are facing be it in project cost estimation, discounted cash flow analysis or, well, anything really. But this is only one facet of risk and decision assessment, specifically dealing with the descriptive statistical output from a simulation. What of the decision evaluation component? Why aren’t more of my customers analysing the decisions they make, or better yet actually optimising them? I have a few ideas why.

If you’re in business you have to make decisions. Big ones, little ones, yes/no, multiple state and continuous value decisions. Decisions that impact other decisions in simple or complex dependency structures. But are you making the best decisions possible? I’m sure important decisions aren’t being made completely randomly (I hope!) but I see many companies who rely completely upon qualitative techniques for their decision making (experience, gut feel, etc.) which of course means optimality is no more than a hoped for outcome rather than something that is actually being worked towards.
Firstly the decision model must be identified and then quantified, and this can be a difficult task. There is a level of modelling aptitude necessary for effective modelling that goes beyond merely knowing Excel and its functions, and into the construction of logical mathematical descriptions of possibly complicated processes. Relevant decisions need to be identified and the impact of those decisions combined into a formula that can be mathematically optimised. A critical component to all this is the knowledge that spreadsheet models can actually be optimised, and that in cases where Excel’s Solver fails there are Palisade products (Evolver and RISKOptimzer) that can perform optimisations under virtually any circumstance.

I too used to focus on Monte Carlo simulation rather than decision evaluation, and this was mainly a product of the clients I was dealing with almost exclusively when I first worked for Palisade. In my next blog I’ll tell you why that changed and also get a little more into the nuts and bolts of optimisation.

Rishi Prabhakar
Trainer/Consultant

Why is it that most of the high profile projects managed by the government in the UK all ultimately become beset by problems? A number of projects jump to mind – the Millennium Dome, Wembley Stadium and currently the NHS IT. All three have been plagued by developmental delays and financial mismanagement.

Recently, yet another worthy, but ambitious project has been announced – the North-South high speed rail line to connect London to Scotland. One wonders if the government undertakes detailed quantitative project risk analysis for its infrastructure initiatives?

A good example to highlight in this context is ENGCOMP, a Saskatchewan-based engineering consulting firm that has worked with the Canadian Department of National Defence (DND) to help define budgets for the fourth phase of construction of its Fleet Maintenance Facility at Canadian Forces Base Esquimalt in Victoria, British Columbia. Using @RISK, a Monte Carlo simulation tool, ENGCOMP helped the DND define and secure budget approval from the Federal Government’s Treasury Board. The consultancy firm was able to estimate the impact of the variability and uncertainties pertaining to risks, costs and scheduling. This assessment enabled it to estimate the project risk budget or the risk reserve and schedule contingency, which were both factored in when defining the total project cost of the infrastructure project.

The fact is, in the world of business, risk is inherent and unavoidable. Whilst one cannot completely control risk, one can certainly help reduce uncertainty, greatly increasing the chances of project success. For instance, a key finding of the project risk analysis conducted by ENGCOMP was that, taking into account all the risk and uncertainties on the project, there is an 85 per cent chance that the Fleet Maintenance Facility project will be completed in January 2014. A fairly positive result for the DND, given the scale and complexity of this project in question.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Where I left off last time was lamenting the use of Monte Carlo simulation to create a single value (statistic etc.) from a model. It might still not be clear why this is anathema to me, so here goes.

A simulation is not a number. It’s not one possible (future) outcome – that’s a scenario. Monte Carlo simulation is a methodology for understanding one’s exposure to outcomes not situated close to the central tendency of the process/project in question. Note the plural “outcomes”. Risk analysis, when done properly, should let you know essentially all possible outcomes and how likely they are for your model. Output from a simulation can include a plot of means (over time), or P5s, or P95s, or the mean ± one standard deviation or any number of statistics. But that’s not plotting a simulation! Let’s not give a minimalist graph too much credit.

Such statements also perpetuate the idea that simulation is only used for creating means (or other centrally tending statistics) and ignores the wealth of information available. Risk simulation software exists to help you do risk analysis which must include not only several statistics but also sensitivity information. It is all too easy to turn a risk assessment into a hunt for a regularly asked for percentile (such as the P90) and there ends the task. I see this a lot, especially in project cost estimation where the pressure both from management and regulatory bodies is to accurately estimate some large percentile. Once found there is usually scant further risk analysis.

Nothing good ensues. When risk analyses are run “to get ‘the’ number” they become simply another box to tick in a process and ultimately any benefits (perceived or actual) will be forgotten and lost to the ages. The notion of context is also lost. No single number by itself really means anything, or at least shouldn’t mean anything to a decision maker. I have often heard phrases like “the model returned/gave $1.2m” followed by an audience nodding in agreement. Huh? Which statistic are you talking about there, and how about reporting a few other numbers around it to place that $1.2m somewhere meaningful?

In the next installment I will look further into this issue of context and hopefully prove the necessity of an holistic approach to understanding and reporting simulation results.

» Part 1

Rishi Prabhakar
Trainer/Consultant