Risk Analysis Software

This Thursday, 28 January 2010 at 11am ET, Dr. William Strauss, President of FutureMetrics, will present a free live webcast entitled, "Simulating the U.S. Economy: Where will we be in 100 years?" Sign up now to attend the webcast.

There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

To see where this growth imperative will take us we first have to see how we go to where we are today. This free live webcast first models the 20th century. The model is both complex and simple. The basic schematic of the model’s relationships is easy to understand. Furthermore, the core of the model is a simple production function that combines capital, labor, and the useful work derived from energy to generate the output of the economy. Complexity is contained in the solutions to the internal workings of the model. What is unique is that there are no exogenous economic variables.  Once the equations’ parameters are calibrated, setting the key outputs to “one” in 1900 results in their time paths very closely predicting the U.S. GDP and its key components from 1900 to 2006. 

The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? From 1900 to 2006 there are periods in which there was parameter switching. (The optimal parameters and the years for the switching were found using a constrained optimization technique.) That suggests that in the future there will also be changes. The experiment uses @RISK’s features (risk analysis software using Monte Carlo techniques) to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization using Monte Carlo simulation) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

----
William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics).

» Register now for this FREE live webcast
» View archived webcasts

In my last blog I mentioned a ‘fact’ about data that came up during a recent public training course (Decision-Making and Quantitative Risk Analysis). This fact stuns me every time I think about it, and certainly floored me the first time I encountered it. So many companies just don’t have it.

Data, that is. Historical data from completed projects, sometimes billion-dollar projects, is simply not collected especially in resources and infrastructure cost estimation. Instead every risk is re-estimated from scratch in every new project based entirely upon an estimator’s recollections or guesses. This is not a suggestion that estimators don’t know what they’re talking about, rather that the benefits of adding historical data to the analysis far outweigh the cost of gathering the information in the first place.

I first worked in the banking sector, hence my surprise to learn of this lack of data storage in certain areas of risk analysis. Project cost estimation, especially in resources and infrastructure – I’m talking to you. In financial circles there are literally millions of data points collected daily across the entire organisation. Gathering data (and then analysing it for some benefit) is simply ‘what we do’, and this process isn’t challenged. Some of the data is quite ‘small’, such as the number of seconds a particular caller was kept on hold before being answered, and others are quite ‘big’, such as multi-million dollar losses due to fraudulent activities. Regardless, it’s all kept in the knowledge that information is power – in this case the power to make intelligent decisions in the future.

How can you judge the efficacy of an estimation process (workshops etc.) if you don’t track the final observed outcomes specifically to make such a judgment? Well, you can’t. And that leaves your company’s risk and decision assessment process in limbo. Without measurement there can be no process improvement or corporate learning. Are you ‘passing’ or ‘failing’ with your use of Monte Carlo simulation via risk analysis software?

Generally the observed outcomes for risks in models will be near the estimated value, and this is to be expected. However the main role of risk analysis is to adjudge exposure to the unexpected. Far too many cost estimation models have very little volatility in their line items. I am very curious to know just how often the realised value of a given line item is outside the range of “possible values” as defined in the model. And what about the total project costs overall? This hints at and leads to the big question which is what could/should be done with such data if it were to be recorded?

I shall address these questions in the next blog. I know you’re excited to find out!

Rishi Prabhakar
Trainer/Consultant

Today there is a heightened appetite for risk management due to global economic circumstances. But risk management has always been an intrinsic aspect of business to a higher or lesser degree. However, in the current technology-led business environment, the use of software to effectively manage risk makes logical sense. It provides a level of sophistication that the traditional processes simply cannot offer. Let me explain why.

Risk management essentially involves three stages – identification, quantification, and the on-going management of risks. In reality, these stages are not completely distinct from each other, with each stage influencing and informing the others. For example, an initial quantification of risks may lead to the conclusion that some of the identified risks are in fact not serious enough to warrant further consideration, or that the original description of the risk was not sufficiently precise for meaningful risk management measures to be put in place.

Each of these stages can benefit from the use of supporting risk modeling software. For instance, Microsoft Excel can be used to create a risk register, i.e. a database that records the risks identified, the assessment of the likelihood and impact of each of these risks, the mitigating actions that have been planned, and the assignment of responsibilities for these actions. However, there are many other software tools available, each designed for a specific purpose and focus. To illustrate, enterprise-wide risk management software focuses on the creation of integrated and holistic risk management systems, whereas Monte Carlo simulation and decision tree software place their emphasis on enhancing the quantitative analysis of risks.

The selection of the appropriate risk analysis software should involve very careful thought. The right decision can lead to a very effective implementation, whereas the wrong decision may result in a large amount of wasted investment.

There are some key considerations to bear in mind when selecting the risk modeling software. Choosing software based on how many staff will genuinely be required for the day-to-day risk management process is crucial. It is easy to select software based on the ideal situation that there will be a wide staff involvement in the risk management process. In reality, this may not be possible, potentially resulting in a cumbersome and inflexible solution being chosen over a more stand-alone and flexible application.

Similarly, knowing the level of risk quantification required is important. In fact, best practice risk management now involves the use of quantitative techniques, often using Monte Carlo simulation. When correctly conducted, the process of quantifying risks is rigorous and structured, can expose hidden or biased assumptions, as well as provide a more solid rationale upon which to base the major decisions.

Finally, determining the extent of on-going risk management needed for your business can assist with software selection. 

Needless to say, any software application will be most successful when used by appropriately trained and motivated staff, and when used as a supporting tool within an overall risk management process. Software is not a replacement for process.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Following on from the resounding success of the last Palisade Risk Conference in London, which attracted over 110 attendees from industry and academia, the 2010 Palisade Risk Conference will be taking place on April 14th-15th. The location for this event will again be the Institute of Directors on Pall Mall, London, and already there are a number of exciting presentations confirmed from the likes of Unilever, Pricewaterhouse Coopers and Halcrow.

The 2010 Palisade Risk Conference will be a two-day forum which will cover a wide variety of innovative approaches to risk and decision analysis. Featuring real-world case studies from industry experts, best practices in risk and decision analysis, risk analysis software training, and sneak previews of new software in the pipeline, the event is also an excellent opportunity to network with other professionals and find out how they’re using Palisade risk analysis solutions to make better decisions.

Call for Papers

If you have an unusual or interesting application of Palisade software which you would like to present, please send a short abstract to cferri@palisade.com. The closing date for abstracts to be submitted is Friday, 11th December, 2009.

@RISK allows you to use historical data to fit data to a probability distribution. The process is very simple: first select the range where the data is located, and then select the Distribution Fitting button. @RISK will guide through the fitting process where you can select a variety of statistical tests such as Chi-Square, Anderson-Darling, Kolmogorov Smirnov, and the Root-Mean Squared Error. View a short tutorial about Distribution Fitting in risk analysis models below.



While the Distribution Fitting functionally is very useful, in some real life cases we need to fit hundreds of distributions, or create filters for certain date ranges or conditions. If we are to do this manually, the fitting process can be overwhelming. A batch fitting function streamlines the process in your risk modeling software.

Palisade’s Custom Development Team  has been helping many of our customers automate this process using the @RISK Developers Kit. With a custom batch fitting add-in, we are able to extract information from external databases and organize data so that the fitting process can be done automatically. The resulting distributions can be dropped with ease into risk analysis models.

If you are interested in this type of consulting support for risk analysis models, please let us know. Feel free to contact your Palisade sales representative.





>> View @RISK tutorials

Javier Ordóñez, Ph.D
Director of Custom Solutions

When modeling risk events, it is common that several events could affect the same cost element of a project. During the simulation, two or more risk events can occur at the same time. The question becomes how to calculate the total impact. This type of modeling technique is very common and often needed in project risk analysis, contingency and mitigation studies, reserve estimation and production forecasting.

A common practice is to aggregate the total impact. However, this simplistic approach might not be correct since it ignores the compounding effect of multiple occurrences. For example, if we have two risk events occurring and their respective impacts are a 20% and 40% cost increase, an additive model will calculate the total impact as 20%+40%=60%. On the other hand, a multiplicative (compound) model will calculate the total impact as 1.2 x 1.4 = 1.68; here the total impact is 68%. In other words, the impact of these risk occurrences will be greater than the summation of the individual impacts, which in many cases makes sense.

Using the table below we will show how you can use @RISK (risk analysis software using Monte Carlo simulation in Excel spreadsheets) to can calculate the distribution of the total impact that can affect the cost of a project activity. You can see that in this example we also model the opportunity of a cost reduction.





To model the % cost increase and reduction we use a Triangular distribution. Here we say that the maximum impact is the value in column F (the user can use other distributions or parameters as needed). With this logic the distribution of the impact of Risk 1 looks like:



and the total impact distribution results in:



The following table contrasts the results obtained using a compound and an additive model:



Something to note in the table above is that in the opportunity side, the maximum cost reduction using the compound model is less conservative than the additive model.

As seen here, coding this type of model is not difficult. My suggestion is that when you have multiple occurrences of risk events you explore better alternatives than the additive model. As always, your comments are more than welcome.

Dr. Javier Ordóñez
Director of Custom Development

The latest version of @RISK (risk analysis software, Monte Carlo software for Excel), has a new graphing tool that will let you create a scatter plots using any two outputs of your risk model. 

For example, if you construct an integrated schedule and cost risk model you can easily evaluate the distributions of the duration of the project and its cost. However, you might  also be interested to know the probability of meeting a target duration and cost at the same time.

The duration and the cost of the project can be defined as outputs of the Monte Carlo simulation model. Once the simulation is run, you can assess the total cost and duration at the 85th percentile as seen in the figures below. Here you can see that the duration at this percentile is 1500 days and the cost is $11.35 Millions.


Next, you can create a scatter plot with these two outputs and locate the delimiters at the target values for both variables as shown below. Here you can see that there is only a 2.9% chance that those two targets will be exceeded and there is a 72.8% that both targets will be met. You can also evaluate the probability that only one target is met.

I hope that this new tool can help you to understand in a better way the interaction between a two outputs of your @RISK risk analysis model.

Dr. Javier Ordóñez
Director of Custom Development

Dr. Clayton Graham is an adjunct professor of Statistics and Economics at DePaul University. He holds senior positions with the Chaos Group, Inc. and Analytical Advantages, LLC where he functions as a management consultant specializing in analytical and graphic econometrics.

He will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for Clay Graham's case study below, and see the full schedule for the Conference here.

Capitalizing Upon Market Inequities:
A Game Plan for Successful Sports Wagering

Sports wagering brings two separate "markets" together. First is the production market or the game itself.  The second is the wagering or betting market. As a matter of practicality, the wagering market is itself in balance, i.e., bet clearing is covered through the process of adjusting the cost-payout ratio (the line). Betting lines are translated into an expected probability of winning. This resultant probability is frequently inconsistent with the probability of the team actually winning. 

Hence, the opportunity to capitalize upon the dichotomy between the inequities of the production and gaming markets will be detailed and quantified. The presentation will include:

• Fundamentals of gambling lines and odds,
• Identification of key metrics,
• Methods of production modeling baseball and basketball
      (similarities and differences),
• Integration of economics (investment) with production,
• Economics of decision making.

Principal Palisade software utilized includes: StatTools (statistical analysis toolkit for Microsoft Excel), @RISK (risk analysis software using Monte Carlo techniques in Excel) and Evolver (genetic algorithm optimization in Excel). The presentation will have a heavy graphic and visualization emphasis. Theoretical statistics will be tightly tied with pragmatic realities of game modeling and economically based decision making. 

Specific quantification will consist of:

• Probabilities of winning a game,
• Measurement bias of officials,
• Quantification of player performance,
• Expected values of return on investment,
• Sports gambling optimizing algorithm.

Examples of actual results for current and past seasons along with predictions will be provided. 

In short, it’s "Card Counting" for sports!

More information about this project can be seen at Baseballwon.com.

The recent Wall Street Journal article “Odds-On Imperfection: Monte Carlo Simulation” asserts that Monte Carlo simulation did not predict the market crash, and cites a chorus of critics calling for a fix to the technique. The article equates the technique of Monte Carlo simulation with the models that are using it – two very different things. For instance, the article states, “These models were supposed to help quantify and manage the risks of mortgage-backed securities, credit-default swaps and other complex instruments. But given the events of the past couple of years, it appears that the models often gave big institutions, as well as small investors, a false sense of security.”

This is true – the models for decision making under uncertainty gave a false sense of security. But that’s because the assumptions underlying the risk analysis models were flawed, not because the technique of Monte Carlo simulation was problematic. Monte Carlo simulation is simply a mathematical technique that recalculates many different possible scenarios – but only within boundaries set by the user.  You can’t change the underlying math behind these “what-if” calculations.

The article comes close to making this distinction in one sentence: “Critics emphasize that the problem isn't Monte Carlo itself, but the assumptions that go into it.”  It then goes on to describe efforts by firms to include “fatter tail” distributions that more accurately reflect the probability of extreme events occurring as an effort to improve Monte Carlo simulation.  Tools like @RISK (risk analysis software add-in for Microsoft Excel) allow complete control over the definition of many dozens of distribution types, enabling users to create as fat a tail as they want. While these efforts make sense, it should be made clear that these are changes to underlying model assumptions, not changes to Monte Carlo simulation itself. To equate Monte Carlo simulation as a technique with the probability distributions people decide to use is to equate a carpenter’s choice of nails with his hammer.

Finally, the article cites the need to run tens or hundreds of thousands of scenarios, instead of just 100 or 1000.  This too is user-defined, and tools like @RISK can run as many scenarios as desired.

Randy Heffernan
Vice President

In the first three entries of this series (see links below), we discussed the use of risk analysis to:

1. establish the average outcome, the average being a crucial reference quantity in decision-making in financial contexts (but a value that is not shown in a static model generally), and
2. establish the variability of the outcome, such variability being of high importance in both non-financial and financial decision making in practice, and
3. create a revised and more complete model including event risks, risk mitigation measures, and implied optimization of risk management.

Here we point out that risk modelling using Monte Carlo simulation can be used to capture aspects of modelling that are hard or impossible to capture using other techniques. For example:

1. Dependency relationships, such as correlated sampling and parameter-dependency between distributions
2. Simulation establishes the probabilities of outcomes (not just their possibility, as would a sensitivity analysis), it allows for the simultaneous variation of three or more risk factors (whereas Excel DataTables do not), and hence can deal with the large number of possible combinations for input variables.
3. Some situations are inherently stochastic, and cannot be modeled using sensitivity analysis; these include especially the modelling of contingent claims (e.g. options, real options, profit share agreements, incentive schemes penalty clauses) etc.

The DecisionTools Suite is a complete toolkit for decision making under uncertainty, including @RISK risk analysis software.

See the previous posts in the "Why Use Risk Analysis?" series:
» Part I
» Part II
» Part III

Dr. Michael Rees
Director of Training and Consulting

When the economy takes a turn for the worse, tough times call for cutbacks. Cutbacks might include extras you don’t really need, goods that haven’t yet outlived their usefulness, or services you can perform yourself. Just because you can – and perhaps should – do without some of the ‘luxuries’ does not mean everything should be cut from your budget. Expert advice is one of those areas – and it doesn’t have to break the bank.

Weighing important decisions demands advanced analytics and informed insights to uncover the value of one choice over another. With quantitative analysis, you can gain valuable insights into underlying risks and their implications. Taking that information forward, you can formulate effective responses to those risks. Statistical analysis in Microsoft Excel is useful, but without the ability to account for risk and uncertainty, a static model adds limited value.  George Box’s adage “All models are wrong. Some models are useful” is an apt perspective viewed through static analysis. An essential extension to the basic Excel model is quantitative risk analysis.

Palisade’s training and consulting services are cost effective ways to extract more from your risk analysis software. In short order you can gain better control over your simulation models through training, or you can work with the experts who developed these tools to aid you in efficient modeling of your given situation. Bespoke models provide you with information relevant to your organization. Add efficient application of @RISK (analysis with Monte Carlo simulation) and RISKOptimizer (Monte Carlo simulation with optimization for decision making under uncertainty) or PrecisionTree (decision analysis in Microsoft Excel using decision trees and influence diagrams) and you get much more. Then you’ll find out just how useful models can be.

Thompson Terry
Palisade Training Team

In an article this week in the online publication TheStreet, Taylor Smith takes aim at online calculators for retirement planning.  He says the problem with them is that most of them are based on Monte Carlo software, and then goes on to make some broad and inaccurate statements about the characteristics about Monte Carlo simulation and the ways it skews risk assessment.  

For starters, he quotes an investment "expert," who is, coincidentally, president of concern that produces risk analysis software not driven by Monte Carlo software, to the effect that good financial planning should take into account, more than market returns--including taxes, income, and expenses.  This implies the Monte Carlo simulation is not capable of factoring these elements into its predictions.  Nothing, as the a recent risk analysis model offered by both the Society of Actuaries and the Casualty Society aptly demonstrates, could be farther from the truth.  This model, developed by professors of finance and mathematics at Illinois State University and the University of Illinois Champaign-Urbana, helps pension and insurance planners to to forecast not only projected market returns but the effects such critical economic factors as interest rates, equity price levels, inflation and unemployment rates, and real estate prices.

My point is that a Monte Carlo simulation is what you make it.  It can be very simple and limited to one or two economic factors, or it can be a complex mix of many factors.  If the online retirement calculators are too simplistic to usefully account for reality, their builders have plenty of room to improve them.     

When using @RISK (risk analysis software for conducting Monte Carlo simulations in Microsoft Excel), one of the output graphs is a tornado graph. Such graphs have their most direct interpretation for linear models with independent input distributions, such as in most typical cost budgeting models. In these cases, the regression coefficients provide a measure of how much the output would change if the input were changed by one standard deviation (the correlation coefficients provide a broadly similar measure, but are slightly different as is covered in another posting). In @RISK5, the “mapped values” feature shows the absolute figures i.e. the absolute change in the output as each input is changed in this way.

For models which have dependencies between the input distributions (e.g. correlation or parameter dependencies) and models where the output behaves in a non-linear way with some of the inputs, these statements hold either with some qualification or may not hold at all. In such cases, the interpretation of the coefficients will in general be specific to the nature of the model. For example, in linear models with correlated input distributions, the regression coefficient will still provide a measure of how much the output would change if the input were changed by one standard deviation, but only assuming that such a change could be implemented independently i.e. without affecting the other variables.

» Watch a video demonstrating Tornado Graphs in @RISK

Dr. Michael Rees
Director of Training and Consulting

In this post, we're looking at the components of a risk analysis performed to estimate costs of a construction project. These analyses can be performed in @RISK, Palisade's software for risk analysis using Monte Carlo simulation. @RISK is an add-in to Microsoft Excel.

When dependence exists, the estimated probability density functions (PDFs) of the cost components variables are the marginal PDFs of the joint PDF of the component variables. The PDFs alone are not sufficient for estimating the PDF of total project cost. When positive dependence exists, the effect of assuming independence is underestimation of the variance of the system variables. Under the independence assumption, the single figure estimate of the system variable is almost guaranteed to be exceeded if the summation of the estimates is a large number of small subsystem variables; this seems to contradict the conventional wisdom that subdivision of construction projects into smaller work packages facilitates cost estimation and improves accuracy.

In construction cost estimating the assumption of independence is usually adopted due to the difficulty of modeling dependence. The extent and nature of interdependence does not depend only on the specific project characteristics but also on the number of cost components and the way they are defined. In general, the larger the number of components, the higher the chance that dependence exists. One way to avoid correlation is to divide the system into fewer subsystems or by grouping correlated or independent subsystems into a single subsystem; however this strategy might complicate the estimation of subsystems if they are too large or complex.

» Read about construction consultancy Pantektor's use of @RISK
» Pantektor AB

Real options are the flexibilities that are inherent in general business or other decision situations. In general, a real option is present in any decision situation involving a decision-chance-decision sequence; the possibility to (at the second decision) select from a range of different decision possibilities after the occurrence of the chance event may alter the choice of the decision earlier on in the sequence (and/or increase its value). The extra value created by this flexibility is sometimes described as a real options value.

Real options analysis concerns itself with analysing such flexibilities. On some occasions it may be desired to value such flexibilities explicitly. On others, the valuation is not explicitly required and the analysis concerns itself mostly with making the correct decisions and planning risk response or mitigation actions. The topic has links to financial market options, as well as to traditional net present value analysis.

A more detailed description of this topic, with example models using Excel, @RISK (software for risk analysis using Monte Carlo simulation) and PrecisionTree (decision trees in Microsoft Excel) can be found in Chapter 5 of my book Financial Modelling in Practice (John Wiley & Sons, 2008. ISBN-13: 978-0470997444).

Dr. Michael Rees
Director of Training and Consulting

When using @RISK (Monte Carlo software for risk analysis and risk assessment), a user may choose between the Monte Carlo (MC) and Latin Hypercube (LH) sampling types.  LH sampling involves a stratification of the input distribution i.e. the cumulative curve is divided into equal intervals on the cumulative probability scale (0 to 1.0).  In theory, LH sampling would create a more representative sampling of the distribution.  It would avoid potential non-representative clustering of sampled values (particularly when small sample sizes are drawn i.e. a small number of iterations is used), and would also ensure that tail samples of the input distributions are drawn (e.g. for 1000 iterations, exactly one value above the P99.9 would be drawn, whereas for MC sampling either none, one or several samples may be drawn).  In general, LH sampling would be favourable when testing and developing a model (running only a small number of iterations), or when running a model that is so large that only a small number of iterations can be conducted.  It can also be used to force the sampling of tails of distributions, although it should be remembered that LH stratification applies to each individual input variable, and it would not force the simultaneous sampling of tail values for more than one input.

Dr. Michael Rees
Director of Training and Consulting