New Approaches to Risk and Decision Analysis

Wednesday, March 17, 2010 by DMUU Training Team


Risk analysis and decision-making tools are relevant to most organisations, in most industries around the world.  This is demonstrated by the speaker line-up at this year's European User Conference, an event at which we believe it is important to bring together customers from a wide range of market sectors.

We are holding 'New Approaches to Risk and Decision Analysis' at the Institute of Directors in central London on 14th and 15th April 2010.  As with previous years, the programme aims to provide everyone attending with practical advice to enhance the decision-making capabilities of their organisation.  Customer presentations, which offer insight into a wide variety of  business applications of risk and decision analysis, include:
  • CapGemini: Faldo's folly or Monty's Carlo – The Ryder Cup and Monte Carlo simulation
  • DTU Transport: New approaches to transport project assessment; reference scenario forecasting and quantitative risk analysis
  • Georg-August University Research: Benefits from weather derivatives in agriculture: a portfolio optimisation using RISKOptimizer
  • Graz University of Technology: Calculation of construction costs for building projects – application of the Monte Carlo method
  • Halcrow: Risk-based water distribution rehabilitation planning – impact modelling and estimation
  • Pricewaterhouse Coopers: PricewaterhouseCoopers and Palisade: an overview
  • Noven: Use of Monte Carlo simulations for risk management in pharmaceuticals
  • SLR Consulting: Risk sharing in waste management projects - @RISK and sensitivity analysis
  • Statoil: Put more science into cost risk analysis
  • Unilever: Succeeding in DecisionTools Suite 5 rollout – Unilever's story
We will also look at the recently-launched language versions of @RISK and DecisionTools Suite, which are now available in French, German, Spanish, Portuguese and Japanese.  Software training sessions will provide delegates with practical knowledge to ensure they can optimise their use of the tools and implement business best practise and methodologies.

With over 100 delegates from around the world attending, the event is also a good opportunity to network and knowledge-share with risk professionals from around the world.

» Complete programme schedule, more information on each presentation,
   and registration details



Rumors of Death

Monday, March 15, 2010 by Holly Bailey
Allan Roth, who writes a blog for CBS Money Watch called "The Irrational Investor," recently asked his readers a rhetorical question: Is Financial Monte Carlo Simulation Dead? Since rhetorical questions demand an answer in less time than it takes the questioner to draw breath, Roth obliged. 
 
While expressing sympathy for the investors who were victims of poor risk assessment and forecasting when the financial markets shook themselves down to rubble in 2008, Roth is taking a very politely defensive swing at one of the many critics of risk analysis who have turned up the volume since then--one Jim Otar of Otar Retirement Solutions and the author of Unveiling the Retirement Myth.  

Roth is an experienced user of Monte Carlo software who knows the pitfalls of overoptimistic assumptions.  He says he finds 99 percent of the Monte Carlo models he's see over the years to be inadequate because of this flaw.  Jim Otar, for his part, finds other flaws as well: in the generation of randomness and trends and in the sequence of returns. Otar's modeling method does not rely on randomness but on a century's worth of historical data. 
 
Our two worthy opponents put their models up against one another in a match that crunched identical inputs.  Their models produced very, very similar results, apparently satisfying each analyst as to the superiority of his method.  But while Roth said nice things about Otar and his model, he pointed out the limitations of relying on historical information alone. In other words, he doesn't concede.
 
For any kind of retirement planning models, he says, the cure to flaws is conservative input. Then he giddily sends his readers to one of those rudimentary online Monte Carlo calculators that investment firms love to offer their clients. 
 
Rumors of this death are greatly exaggerated.  

How can the UK public services prepare for unpredictable, extreme weather?

Friday, March 5, 2010 by DMUU Training Team
The UK Met Office is not going to ‘live down’ its weather forecast of a ‘barbeque Summer and a mild Winter’ for 2009, anytime soon. There was ample rain through the Summer, the Cumbrian region saw severe flooding in November and now the nation is gripped by sub-zero temperatures not experienced for more than 30 years.

The inaccurate weather forecast is not a criticism of the Met Office. Forces of nature cannot be controlled, but these severe weather conditions do highlight the need for a more risk-led approach to public service planning. As we are seeing, the lack of planning to combat the current Arctic conditions engulfing the nation has thrown the country in turmoil, not to mention the substantial losses incurred by businesses. 

Global Warming is now often touted as the reason for such vagaries in weather, which according to environmentalists is set to intensify in the coming years. There is a very strong case for the government to undertake a scientific, risk-led approach to assess the potential effects of extreme weather, so that the required planning and realistic fund allocation can be made to deal with unforeseen weather situations. 

For instance, Halcrow Group Ltd, specialising in providing planning, design and management services for infrastructure development, works very closely with the UK Environment Agency on its Flood Defense programme. It conducts risk analysis on several of the Agency’s projects, using Palisade’s @RISK. Through flood risk management, the UK’s Environment Agency can reduce the probability of flooding from rivers and the sea through the management of land, river systems, and flood and coastal defenses. This also works to helps to reduce the damage floods can do through effective land use planning, flood warning and emergency responses.

There is now a dire need to extend this risk analysis-based approach beyond just flood defense, so that pre-emptive actions can be taken to reduce the adverse impact of extreme weather on the nation.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

New business planning – measuring feasibility

Tuesday, February 23, 2010 by DMUU Training Team
The latest Business in Britain survey from Lloyds TSB Commercial shows that the UK's commercial enterprises are regaining confidence.  The six monthly report charts the performance of 1,732 UK companies and their views on prospects for the coming year. Its most recent business confidence shows that expectations for both sales and orders have started to recover. The balance of firms anticipating an upturn in sales has climbed to 21% - from just 1% six months ago.   And hopes for orders are also looking brighter. The balance expecting order levels to rise over the coming six months has climbed to 23%, from just 6% in the last survey.

But companies planning major new business drives for 2010 would do well to follow the example of Thales UK, which uses @RISK  to enable it to assess commercial feasibility of potential new business wins. @RISK's in-depth risk analysis ensures the leading provider of mission-critical electronic information systems for aerospace, defence and security markets around the world, is fully informed when making business-critical decisions.

Thales operates in a highly competitive environment, with technologically advanced countries presenting tough opposition when it tenders for contracts. It must continually develop highly sophisticated equipment that is robust and failsafe to meet the stringent demands of its customers. Bringing products of this calibre to market is costly in terms of time and resource, so for every competitive new business opportunity, Thales must be confident that it has a reasonable chance of success.

Using Monte Carlo analysis to show all potential scenarios and the likelihood that each will occur, @RISK enables Thales to calculate the competitiveness of complex markets, measure probabilities for project costs, quantify rate of return, and even account for the effects of cumulative business, thereby providing decision-makers with the most complete picture possible.  From this risk analysis, Thales can make an informed decision on the commercial viability of the potential new business offered.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Free Webcast this Thursday: "Simulating the U.S. Economy: Where will we be in 100 years?"

Tuesday, January 26, 2010 by DMUU Training Team
There is an assumption that drives all of our expectations for how our economy will be in the future.  That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

In this free webcast, Dr. William Strauss models the next 100 years, based on the last century's data. The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? The experiment uses @RISK’s risk analysis and Monte Carlo techniques to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization software) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

Palisade is pleased to host this presentation from Dr. William Strauss.

William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics). Read more of Dr. Strauss' bio here.

» Complete abstract of "Simulating the U.S. Economy: Where will we be in 100 years?" 
» Register now (FREE)  
» View archived webcasts

Free Live Webcast this Thursday: Simulating the U.S. Economy: Where will we be in 100 years?

Monday, January 25, 2010 by DMUU Training Team
This Thursday, 28 January 2010 at 11am ET, Dr. William Strauss, President of FutureMetrics, will present a free live webcast entitled, "Simulating the U.S. Economy: Where will we be in 100 years?" Sign up now to attend the webcast.

There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

To see where this growth imperative will take us we first have to see how we go to where we are today. This free live webcast first models the 20th century. The model is both complex and simple. The basic schematic of the model’s relationships is easy to understand. Furthermore, the core of the model is a simple production function that combines capital, labor, and the useful work derived from energy to generate the output of the economy. Complexity is contained in the solutions to the internal workings of the model. What is unique is that there are no exogenous economic variables.  Once the equations’ parameters are calibrated, setting the key outputs to “one” in 1900 results in their time paths very closely predicting the U.S. GDP and its key components from 1900 to 2006. 

The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? From 1900 to 2006 there are periods in which there was parameter switching. (The optimal parameters and the years for the switching were found using a constrained optimization technique.) That suggests that in the future there will also be changes. The experiment uses @RISK’s features (risk analysis software using Monte Carlo techniques) to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization using Monte Carlo simulation) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

----
William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics).

» Register now for this FREE live webcast
» View archived webcasts

Data Issues Part 1

Tuesday, January 12, 2010 by DMUU Training Team
In a recent public training workshop (for @RISK for Excel) I was reminded of an unusual fact regarding data.

Commonly @RISK for Excel is used to fit distributions to historical data for use in risk modelling, and it sure beats wildly guessing obscure parameters. However there are (naturally) a litany of woe-inducing problems with all historical data sets: non-stationary data series, extreme values/outliers, data recording errors, seasonality and heteroskedasticity to name a few. Excessive ‘cleansing’ of the data set is commonly prescribed, but the statistician in me cringes to even type those words! Quality control and transforming the data will help to eliminate most of those problems, but what about outliers?

In the early Naughties I was working for a large Australian bank, forecasting their daily call centre volumes for the purpose of planning staff levels and predicting service levels. A particular call centre averaged 30,000 calls per weekday. Yet on September 12th, 2001, calls dropped to less than 10,000. Along with the rest of the world, Australians were watching the terrorist attacks on television and the internet rather than calling to fix spelling mistakes in their contact details or transfer small sums of money between accounts. But what to do with that data point? Presuming the forecasting model is not intended to include such extreme events as terrorist attacks then the point could simply be filtered out of the data set and not thought of again.

But now consider a process that should include rarer events, such as flood damage or operational risk, as one of the risks in a model. If you have 10 years of good data (say), but the set includes an event that should only occur every 100 years. This level of impact is thus drastically overrepresented in the data and any fitted distribution will be biased toward such extremes. Yet the data point can not be completely ignored as such values can occur and the simulation models must have the capacity to sample such values (though with a reasonable likelihood). In this case the artistry that is fitting distributions to data comes to the fore. The data point could be removed from the set but not from our decision making process.

From the range of distributions that can be selected, the optimal choice should not only represent the remaining data well but also have a tail that samples events in the vicinity of those that have been excluded from the analysis with reasonable probability. No, that’s not always easy to do. But as with many elements of probabilistic modelling it simply must be done in order to provide useful information to decision makers.

Thus the context of the modelling can go a long way to determine the most appropriate steps to take with your data set. If that sounds like a subjective guideline then you read it correctly. Not enough people realise just how important experience and intuition can be in the seemingly prescriptive fields of mathematics and statistics. Fitting distributions to data is no different.

And yet that isn’t the unusual fact I was reminded of in the workshop! But I’ll leave that for Part 2 of my Data Issues blog.

Rishi Prabhakar
Trainer/Consultant

The role of software in risk management

Thursday, January 7, 2010 by DMUU Training Team
Today there is a heightened appetite for risk management due to global economic circumstances. But risk management has always been an intrinsic aspect of business to a higher or lesser degree. However, in the current technology-led business environment, the use of software to effectively manage risk makes logical sense. It provides a level of sophistication that the traditional processes simply cannot offer. Let me explain why.

Risk management essentially involves three stages – identification, quantification, and the on-going management of risks. In reality, these stages are not completely distinct from each other, with each stage influencing and informing the others. For example, an initial quantification of risks may lead to the conclusion that some of the identified risks are in fact not serious enough to warrant further consideration, or that the original description of the risk was not sufficiently precise for meaningful risk management measures to be put in place.

Each of these stages can benefit from the use of supporting risk modeling software. For instance, Microsoft Excel can be used to create a risk register, i.e. a database that records the risks identified, the assessment of the likelihood and impact of each of these risks, the mitigating actions that have been planned, and the assignment of responsibilities for these actions. However, there are many other software tools available, each designed for a specific purpose and focus. To illustrate, enterprise-wide risk management software focuses on the creation of integrated and holistic risk management systems, whereas Monte Carlo simulation and decision tree software place their emphasis on enhancing the quantitative analysis of risks.

The selection of the appropriate risk analysis software should involve very careful thought. The right decision can lead to a very effective implementation, whereas the wrong decision may result in a large amount of wasted investment.

There are some key considerations to bear in mind when selecting the risk modeling software. Choosing software based on how many staff will genuinely be required for the day-to-day risk management process is crucial. It is easy to select software based on the ideal situation that there will be a wide staff involvement in the risk management process. In reality, this may not be possible, potentially resulting in a cumbersome and inflexible solution being chosen over a more stand-alone and flexible application.

Similarly, knowing the level of risk quantification required is important. In fact, best practice risk management now involves the use of quantitative techniques, often using Monte Carlo simulation. When correctly conducted, the process of quantifying risks is rigorous and structured, can expose hidden or biased assumptions, as well as provide a more solid rationale upon which to base the major decisions.

Finally, determining the extent of on-going risk management needed for your business can assist with software selection. 

Needless to say, any software application will be most successful when used by appropriately trained and motivated staff, and when used as a supporting tool within an overall risk management process. Software is not a replacement for process.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Digital Eyes on Alien Life

Wednesday, December 9, 2009 by Holly Bailey
University of Chicago geoscientist Patrick McGuire has big plans for Mars.  Previously he worked on an imager for a Mars orbiter that could identify different types of soil and rock by detecting infrared and other wavelengths, and now he is drawing on that experience to develop a space suit with digital "eyes" and a neural network that rides on the hips of the spacesuit and can sort out living biological material from other matter.
 
The digital eyes will detect and plot colors, and the neural net, which is known as a Hopfield neural network, will compare these color patterns to a database of information previously gathered from that area of planet in order to make an animal-vegetable-mineral determination.  
 
This complex AI system has already been tested at the Mars Desert Research Station in Utah, and McGuire and his colleagues were satisfied that the Hopfield algorithm could learn colors from just a few images and could recognize units that had been observed earlier.
 
McGuire's concept is that a human wearing this neural network could simply walk around the red planet and record every nearby object, rapidly gathering information.  

Obviously, such a clothing item awaits a manned Mars mission.  But in the meantime, why not have the next Rover suit up?  

Free Webcast This Thursday: “Integrated Project Risk Analysis - Structuring the Model Effectively”

Monday, November 30, 2009 by DMUU Training Team
On Thursday, December 3, 11am-Noon ET, Jay O’Connor will present a free Live Webcast about project risk management.

A project risk analysis is only as good as the model that was used to prepare it. It is critical that the model be constructed to reflect the risks specifically associated with the project. The model must be able to accurately reflect the risks associated with schedule, quantities, cost and the residual unmitigated risk items from the qualitative risk analysis. The model should also take into account the interrelationships and dependencies of these items.
This webcast will address these issues and present examples of how results can vary based on the level of detail used in preparing the risk analysis, and will include the use of @RISK, and @RISK for Project.

Palisade is pleased to host Jay O’Connor’s presentation. With over 25 years of experience in the areas of estimating, planning and quantitative risk analysis for international projects, Jay understands the complexities that are associated with identifying and assessing project risks. His experience includes both the owner’s and contractor’s side of engineering and construction projects. He has worked in the upstream and downstream oil and gas industry sectors and the pulp and paper sector. His career has taken him to the United Kingdom, Japan, Indonesia, Malaysia, Singapore and Australia.

» Register now (FREE)  
» View archived webcasts

Two Sides of the Coin

Wednesday, November 18, 2009 by Holly Bailey
Maybe it's because of fallout from the past year's financial crisis, but I have been noticing that almost all the press mention for risk analysis or Monte Carlo simulation is in connection with fending off the bad stuff--loss, adversity, or failure of various kinds.  So it was refreshing to come across a story of decision evaluation being used to analyze the good stuff, that is, innovation and opportunity.
 
In 2008, Dell sponsored a student team from the Tauber Institute at the University of Michigan to compare the opportunity scenarios for designing new laptops that would use emerging wireless technology.  Dell's challenge to the engineering and business students was to determine the most profitable way to approach new laptops for new markets. 
 
Out came the laptops, out came the Monte Carlo software.  In went the inputs--the possible cards, the cost of components, retail discounts vs. direct sales, necessary changes in internal organization.  What was the value-at-risk? An already pretty profit picture from the laptop sales of the previous year. 
 
It was the most positive kind of problem to solve.  And what was the outcome of the team's efforts?  "A Profit-Based Simulation Model for Laptop Planning"-- an optimistic title if there ever was one.  But I suppose the title could have been "Modeling Potential Loss from New Laptop Design."  There were quite a number of good-news scenarios at the institute that year.  I mention the Dell team because of the intensive decision analysis element. 
 
As anyone who does risk analysis is aware, the flip side of opportunity is risk, or maybe opportunity is the upside of risk.  They are always there together, the two sides of chance, but it's great to occasionally see the brighter side of the coin. 

Monte Carlo Meets Monte Carlo

Thursday, November 12, 2009 by Holly Bailey
Monte Carlo is known not only for its casinos and the games of chance that are the namesake of the risk analysis method but also, just as famously, for motor sport. Now, although this has been very little publicized, it appears that Monte Carlo meets Monte Carlo, on a regular basis.
 
A couple of weeks ago, a news item from the United Arab Emirates tipped me off to the fact that Formula 1 racing teams include--in addition to drivers and pit crews--a panel of race strategists. It is the strategists' job to try to plan advantageous responses to any eventuality in a race--rain, wrecks, repairs. Even with the help of computers, forecasting all possible scenarios for a single race is a full-time job, and the F1 strategy teams rely heavily on their Monte Carlo software.  
 
Risk analysis began contributing to F1 strategy as far back as the 1990s and was credited for the McLaren team's 2005 victory in the Monaco grand prix. It is now standard operating procedure. Strategy teams not only pre-play every corner, every curve of a race circuit, but even after the start has sent the cars into high speed, the strategists are responding minute by minute to action on the circuit by running new risk assessments and statistical analyses of emerging scenarios and sending their advice for the drivers via high-speed data links. 
 
Although the race strategist squads haven't received much press, their presence makes perfect sense. After all, who does more and faster decision making under uncertainty than a race driver? And what about the engineers who fine-tune features like aerodynamics and brake design in preparation for a particular race course? And the pit crews on race day? Their function is life or death operations management. 
 
It's a deadly game of chance, the perfect venue for Monte Carlo to meet Monte Carlo. 

Targeted Analyses and Compelling Communication: A Formula for Successful Value Creation in Management Science

Monday, October 19, 2009 by DMUU Training Team
Michael A. Kubica is Founder and President of Applied Quantitative Sciences, Inc. He has over 18 years' experience within the healthcare industry, and has been providing quantitative sciences consultancy since 1999. Michael has extensive experience in providing quantitative decision support solutions for leading pharmaceutical, medical device/diagnostics, and biotechnology companies, addressing a wide range of business issues. Prior to establishing AQS, Michael held the position of Vice President, Operations for Magellan Health Services. During his career Michael has also held positions of Director of Quality Management, Regional Director of Business Operations & Finance, and Hospital Administrator. Throughout his career, Michael employed sophisticated quantitative methods to forecast performance, streamline operations, and improve quality. Michael has an MBA and Master’s of Science in psychology. He serves as Adjunct Professor of Research Design and Statistical Analysis at St. Thomas University in Miami, FL. Applied Quantitative Sciences, Inc. (AQS) is a consultancy specializing in assisting medical device, pharmaceutical and biotechnology companies make decisions under conditions of complexity and uncertainty. They are a market leader in providing simulation and optimization models which are used by industry leaders for the purposes of forecasting, new technology valuation, business and strategic planning, supply chain management, and resource planning.

Mr. Kubica will present a case study later this week at the 2009 Palisade Conference: Risk Analysis, Applications, & Training, 21 - 22 October at the Hyatt Regency in Jersey City (10 minutes by PATH from Manhattan's Financial District).

See the abstract for his case study below, and see the full schedule for the Conference here.

Targeted Analyses and Compelling Communication: A Formula for Successful Value Creation in Management Science

The value of quantitative science projects too often remains unrealized for would-be consumers. Despite flawless analyses, sophisticated reports and dazzling presentations, the message goes unheeded by those who could most benefit: If only they understood how to operationalize the results. The clarity with which quantitative scientists view the practical application of results is often paralleled only by their inability to generate that same clarity in their customers. The result is that good management science is at best ignored and worst, misunderstood (and misapplied). This workshop describes steps we as quantitative scientists can take to foster understanding, generate novel insights and stimulate actionable results with our clients. 

This Week: October 21-22 in NYC

Building on the success of last year’s record-breaking event, the conference will offer a wide range of software training, model building, and real-world case study sessions. Last year, the event drew over 150 practitioners and decision-makers from a broad spectrum of industries. The @RISK and DecisionTools software tracks were more popular than ever. This year, we’re expanding software training with sessions that let you walk through examples and try the tools directly. This will enable you to take some new tips back to the office. Please join us in October for a great opportunity to learn and connect with colleagues.

Contingency Calculation in Cost Risk Analysis

Tuesday, October 13, 2009 by DMUU Training Team
When performing a cost risk analysis study, one of the key results is the amount of extra monetary resources that is to be added to the project cost baseline to guarantee that the budget is not exceeded at a certain confidence level. Good project risk management strategies must take this into account.

After defining the uncertain variables and risk events that affect the cost performance of the project, we can run a Monte Carlo simulation with @RISK to find out what the range of the total project cost is.  Simulation results can help us to explain the risk exposure that we have in the total cost of the project. The most popular statistics are the mean (average cost), the most likely cost, and the 10th and 90th percentiles.




To determine the contingency to be allocated to the project, we need to define what confidence level we would like to achieve: The higher the contingency level, the larger amount of contingency needed. For example, in the figure above, we are reporting the total cost of the project. Here we can observe that we are showing the 85th percentile that corresponds to a total cost of $7.8M (right delimiter).  We can say that there is only a 15% chance that we will exceed $7.8M, or alternatively, we have an 85% chance that the total cost will be less than or equal to $7.8M.  In the same figure we can also see that the 90th percentile of the total project cost is $8.02M.  We can say then that in order to increase our confidence level from 85% to 90%, we will need to add $220,000 to the total cost.

The calculation of the contingency is then accomplished by using the base cost estimate (BE) before the risk analysis was implemented, and the expected cost (EC) of the simulated results.

Some practitioners separate the contingency into two components: engineering allowance, and management contingency.

Engineering allowance (EA) is the difference between the expected cost and the base estimate:

EA = EC – BE

Management contingency (MC) is calculated using the difference between the cost at certain confidence lever (Cp) and the base estimate:

MC = Cp – EC

In our example, our BE = $6.5M; therefore, engineering allowance EA = EC – BE = 6.86M – 6.5M = $0.36M. 

For the calculation of management contingency, we use a confidence level of 85% so Cp(85%) = $7.8M; therefore, MC = Cp – EC = 7.80M – 6.86M = $0.94M.

In many situations, the suggested contingency might be excessive, so the need for a mitigation study is necessary. We can use the sensitivity analysis tool in @RISK to detect the key drivers affecting our total cost. This is valuable information so that we can concentrate our efforts in reducing the impact of risk events and uncertainties to the total cost. Below, we see a tornado graph with the most important drivers. The analyst will then explore the appropriate mitigation strategies and assess their implementation cost. A second simulation can be run to assess the effectiveness of the proposed mitigation plan, and compare the pre-mitigated and post-mitigated cost distributions.




In following blog posts, I will explain how to distribute the assessed contingency to cost elements and identified risk events in project risk management models.

Javier Ordóñez, Ph.D
Director of Custom Solutions

Simulating the U.S. Economy: Where will we be in 100 years?

Friday, September 4, 2009 by DMUU Training Team
William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics).

Dr. Strauss will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for his case study below, and see the full schedule for the Conference here.

Simulating the U.S. Economy:
Where will we be in 100 years?


There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

To see where this growth imperative will take us we first have to see how we go to where we are today. This work first models the 20th century. The model is both complex and simple. The basic schematic of the model’s relationships is easy to understand. Furthermore, the core of the model is a simple production function that combines capital, labor, and the useful work derived from energy to generate the output of the economy. Complexity is contained in the solutions to the internal workings of the model. What is unique is that there are no exogenous economic variables. Once the equations’ parameters are calibrated, setting the key outputs to "one" in 1900 results in their time paths very closely predicting the U.S. GDP and its key components from 1900 to 2006. 

The experiment in this work is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? From 1900 to 2006 there are periods in which there was parameter switching. (The optimal parameters and the years for the switching were found using a constrained optimization technique.) That suggests that in the future there will also be changes. The experiment uses @RISK’s features to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The "doing what you did gets you what you got" scenario leads to a surprising and unsettling outcome. The experiments using @RISK do find a path that works. Obviously if it is not "business-as-usual" that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

Please join us in October in New York for software training in best practicies in quantitiative risk analysis and decision making under uncertainty, real world case studies from risk services consultants and experts, and networking with practicioners from many different fields including oil and gas, pharmaceuticals, academics, finances, Six Sigma, and more.

Integrated Project Risk Analysis

Wednesday, August 26, 2009 by DMUU Training Team
2009 Palisade Conference in New York City Jay O’Connor is a Director at Turner & Townsend Inc. With over 25 years of experience in the areas of estimating, planning and quantitative risk analysis for international projects, Jay understands the complexities that are associated with identifying and assessing project risks. His experience includes both the owner’s and contractor’s side of engineering and construction projects. He has worked in the upstream and downstream oil and gas industry sectors and the pulp and paper sector. His career has taken him to the United Kingdom, Japan, Indonesia, Malaysia, Singapore and Australia.

Jay will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for his case study below, and see the full schedule for the Conference here.

Integrated Project Risk Analysis

When conducting project risk analysis, it is not uncommon for the qualitative risk, quantitative schedule and quantitative cost risk analysis to be conducted separately and kept independent of each other. While some software packages attempt to integrate all three into one analysis, these efforts tend to fall short in one area or another. Turner & Townsend’s approach is to integrate the residual risks and opportunities along with the results from the schedule risk analysis into the cost risk analysis to develop a more fully integrated project risk analysis. The presentation will discuss our approach to risk analysis.

October 21-22 in NYC

Building on the success of last year’s record-breaking event, the conference will offer a wide range of software training, model building, and real-world case study sessions. Last year, the event drew over 150 practitioners and decision-makers from a broad spectrum of industries. The @RISK and DecisionTools software tracks were more popular than ever. This year, we’re expanding software training with sessions that let you walk through examples and try the tools directly. This will enable you to take some new tips back to the office. Please join us in October for a great opportunity to learn and connect with colleagues.


Capitalizing Upon Market Inequities: A Game Plan for Successful Sports Wagering

Thursday, August 20, 2009 by DMUU Training Team
Dr. Clayton Graham is an adjunct professor of Statistics and Economics at DePaul University. He holds senior positions with the Chaos Group, Inc. and Analytical Advantages, LLC where he functions as a management consultant specializing in analytical and graphic econometrics.

He will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for Clay Graham's case study below, and see the full schedule for the Conference here.

Capitalizing Upon Market Inequities:
A Game Plan for Successful Sports Wagering


Sports wagering brings two separate "markets" together. First is the production market or the game itself.  The second is the wagering or betting market. As a matter of practicality, the wagering market is itself in balance, i.e., bet clearing is covered through the process of adjusting the cost-payout ratio (the line). Betting lines are translated into an expected probability of winning. This resultant probability is frequently inconsistent with the probability of the team actually winning. 

Hence, the opportunity to capitalize upon the dichotomy between the inequities of the production and gaming markets will be detailed and quantified. The presentation will include:
  • Fundamentals of gambling lines and odds,
  • Identification of key metrics,
  • Methods of production modeling baseball and basketball
        (similarities and differences),
  • Integration of economics (investment) with production,
  • Economics of decision making.
Principal Palisade software utilized includes: StatTools (statistical analysis toolkit for Microsoft Excel), @RISK (risk analysis software using Monte Carlo techniques in Excel) and Evolver (genetic algorithm optimization in Excel). The presentation will have a heavy graphic and visualization emphasis. Theoretical statistics will be tightly tied with pragmatic realities of game modeling and economically based decision making. 

Specific quantification will consist of:
  • Probabilities of winning a game,
  • Measurement bias of officials,
  • Quantification of player performance,
  • Expected values of return on investment,
  • Sports gambling optimizing algorithm.
Examples of actual results for current and past seasons along with predictions will be provided. 

In short, it’s "Card Counting" for sports!

More information about this project can be seen at Baseballwon.com.

Putting Monte Carlo Software in Reverse

Friday, June 26, 2009 by Holly Bailey
Question for today: What do you get when you run Monte Carlo software back in time? 
 
Answer: You get closer and closer to the wreckage of Air France Light 447.
 
The U.S. Coast Guard's search for the crash site of the doomed Air France plane was the first major test of its "reverse-drift" modeling  program SAROPS (Search and Rescue Optimal Planning System).  Earlier this year I reported on one of its first reality tests, the search for two football players whose boat capsized in the Gulf of Mexico, which apparently took place before the software was formally adopted by the Coast Guard.  For this search, a Coast Guard team in Portsmouth, Virginia, managed the modeling in close cooperation with French and Braziian rescue teams.
 
At last report, the reverse risk analysis was performing admirably.  Starting with the location of the first object sighted in the water, in this case a seat cushion and some smaller debris, team using SAROPS established the location and the immediate wind and current conditions  and then used the history of weather and water since the plane disappeared to estimate thousands of possible paths the seat cushion could have traveled to reach its location.  When the next piece of debris surfaced, its data were fed into the program, and the Monte Carlo software spun out a slightly narrower range of retrospectively possible routes.  
 
Although the reconstruction of the crash location sounds laborious, the simulations are extremely fast.  The software can spin out ten thousand possible routes in fifteen minutes, and as the possible routes of a number of objects begin to converge, they focus with increased probability on the crash site.
 
 A picture--always worth a thousand words--of this clustering of simulated pathways can be found on the Virginian-Pilot website.  If you take a clook at that online graphic, you can see how, in the case of SAROPs, hindsight gets close to twenty-twenty . 

Rethinking Monte Carlo Simulation for Retirement Planning

Friday, June 12, 2009 by DMUU Training Team
A recent article entitled “When Monte Carlo analysis meets a black swan” in Investment News addresses the criticisms Monte Carlo simulation has received for “missing the meltdown.”  The author, Moshe A. Milevsky , notes that people typically seek a single number “answer” from a Monte Carlo simulation, such as the probability of meeting a single retirement planning goal.  Milevsky points out that many Monte Carlo software packages do not include sensitivity or scenario analyses to drill down and determine which variables are really driving the risk inherent in the results.  He proposes what amounts to a stress test – simulating what could happen under likely scenarios, and simulating again under 1-in-100 chance “black swan” disastrous scenarios.  Milevsky wraps up by saying, “Instead of condemning Monte Carlo analyses for missing the meltdown, let's properly harness the full power of stochastic methods to give us tools that provide clear utility.”

I believe Milevsky makes a great point, focusing on the modeling practices rather than the tools themselves in this case.  Monte Carlo simulation tools are very important for applications like retirement planning, but even the best hammer can’t help an unskilled carpenter.  Tools like @RISK include sensitivity and scenario analysis, enabling easy implementation of tests under different scenarios for portfolio value, inflation, longevity, or any combination of these.

Randy Heffernan
Vice President

Don’t Blame the Math

Thursday, June 4, 2009 by DMUU Training Team
A recent article in Bank Investment Consultant criticized the risk analysis method of Monte Carlo simulation for not taking into account extreme events like the stock market crash. According the article, a Morningstar executive states that the “bell-shaped curve that Monte Carlo simulations use” artificially assigns the probability of what happened as zero.  Furthermore, the Retirement Income Industry Association calls for an update to Monte Carlo software simulators to include “a larger number of scenarios that assume greater volatility.”

These arguments demonstrate a fundamental lack of understanding of what Monte Carlo simulation is. The underlying assumption that Monte Carlo simulation itself is somehow to blame fails to recognize that Monte Carlo simulation is simply a mathematical technique that takes into account many different possible scenarios – but only within boundaries set by the user.  You can’t change the underlying math behind  these “what-if” calculations.

When modelers set up retirement planning or financial models, the people doing the modeling must make assumptions about the likelihood of different things happening – like the market crashing, for example.  People may make those assumptions based on historical evidence or expert opinion, but it’s people who make those assumptions – not Monte Carlo simulation software.  People then enter their assumptions into a Monte Carlo simulation model, setting up probability distributions to reflect their chosen likelihoods of occurrence.  If the assumed volatility is insufficient, that is the fault of the modelers, not the simulation itself.

In addition, Monte Carlo simulation does not always “use” a bell-shaped curve.  Uncertainty can be modeled with dozens of different probability distributions, many of them not bell-shaped.  And the call to include more scenarios and volatility can easily be met by existing Monte Carlo software such as @RISK. It’s simply up the user to change the model parameters to look at more possible outcomes. The Monte Carlo simulation package won’t fight it.

It’s disappointing to see esteemed financial organizations such as Morningstar and the Retirement Income Association missing the point. Calling for changes to Monte Carlo simulation itself is not only impossible but fails to recognize the problems with modeling practices that led everyone to miss the crash. 

Randy Heffernan
Vice President