Monte Carlo Meets Monte Carlo

Thursday, November 12, 2009 by Holly Bailey
Monte Carlo is known not only for its casinos and the games of chance that are the namesake of the risk analysis method but also, just as famously, for motor sport. Now, although this has been very little publicized, it appears that Monte Carlo meets Monte Carlo, on a regular basis.
 
A couple of weeks ago, a news item from the United Arab Emirates tipped me off to the fact that Formula 1 racing teams include--in addition to drivers and pit crews--a panel of race strategists. It is the strategists' job to try to plan advantageous responses to any eventuality in a race--rain, wrecks, repairs. Even with the help of computers, forecasting all possible scenarios for a single race is a full-time job, and the F1 strategy teams rely heavily on their Monte Carlo software.  
 
Risk analysis began contributing to F1 strategy as far back as the 1990s and was credited for the McLaren team's 2005 victory in the Monaco grand prix. It is now standard operating procedure. Strategy teams not only pre-play every corner, every curve of a race circuit, but even after the start has sent the cars into high speed, the strategists are responding minute by minute to action on the circuit by running new risk assessments and statistical analyses of emerging scenarios and sending their advice for the drivers via high-speed data links. 
 
Although the race strategist squads haven't received much press, their presence makes perfect sense. After all, who does more and faster decision making under uncertainty than a race driver? And what about the engineers who fine-tune features like aerodynamics and brake design in preparation for a particular race course? And the pit crews on race day? Their function is life or death operations management. 
 
It's a deadly game of chance, the perfect venue for Monte Carlo to meet Monte Carlo. 

Targeted Analyses and Compelling Communication: A Formula for Successful Value Creation in Management Science

Monday, October 19, 2009 by DMUU Training Team
Michael A. Kubica is Founder and President of Applied Quantitative Sciences, Inc. He has over 18 years' experience within the healthcare industry, and has been providing quantitative sciences consultancy since 1999. Michael has extensive experience in providing quantitative decision support solutions for leading pharmaceutical, medical device/diagnostics, and biotechnology companies, addressing a wide range of business issues. Prior to establishing AQS, Michael held the position of Vice President, Operations for Magellan Health Services. During his career Michael has also held positions of Director of Quality Management, Regional Director of Business Operations & Finance, and Hospital Administrator. Throughout his career, Michael employed sophisticated quantitative methods to forecast performance, streamline operations, and improve quality. Michael has an MBA and Master’s of Science in psychology. He serves as Adjunct Professor of Research Design and Statistical Analysis at St. Thomas University in Miami, FL. Applied Quantitative Sciences, Inc. (AQS) is a consultancy specializing in assisting medical device, pharmaceutical and biotechnology companies make decisions under conditions of complexity and uncertainty. They are a market leader in providing simulation and optimization models which are used by industry leaders for the purposes of forecasting, new technology valuation, business and strategic planning, supply chain management, and resource planning.

Mr. Kubica will present a case study later this week at the 2009 Palisade Conference: Risk Analysis, Applications, & Training, 21 - 22 October at the Hyatt Regency in Jersey City (10 minutes by PATH from Manhattan's Financial District).

See the abstract for his case study below, and see the full schedule for the Conference here.

Targeted Analyses and Compelling Communication: A Formula for Successful Value Creation in Management Science

The value of quantitative science projects too often remains unrealized for would-be consumers. Despite flawless analyses, sophisticated reports and dazzling presentations, the message goes unheeded by those who could most benefit: If only they understood how to operationalize the results. The clarity with which quantitative scientists view the practical application of results is often paralleled only by their inability to generate that same clarity in their customers. The result is that good management science is at best ignored and worst, misunderstood (and misapplied). This workshop describes steps we as quantitative scientists can take to foster understanding, generate novel insights and stimulate actionable results with our clients. 

This Week: October 21-22 in NYC

Building on the success of last year’s record-breaking event, the conference will offer a wide range of software training, model building, and real-world case study sessions. Last year, the event drew over 150 practitioners and decision-makers from a broad spectrum of industries. The @RISK and DecisionTools software tracks were more popular than ever. This year, we’re expanding software training with sessions that let you walk through examples and try the tools directly. This will enable you to take some new tips back to the office. Please join us in October for a great opportunity to learn and connect with colleagues.

Contingency Calculation in Cost Risk Analysis

Tuesday, October 13, 2009 by DMUU Training Team
When performing a cost risk analysis study, one of the key results is the amount of extra monetary resources that is to be added to the project cost baseline to guarantee that the budget is not exceeded at a certain confidence level. Good project risk management strategies must take this into account.

After defining the uncertain variables and risk events that affect the cost performance of the project, we can run a Monte Carlo simulation with @RISK to find out what the range of the total project cost is.  Simulation results can help us to explain the risk exposure that we have in the total cost of the project. The most popular statistics are the mean (average cost), the most likely cost, and the 10th and 90th percentiles.




To determine the contingency to be allocated to the project, we need to define what confidence level we would like to achieve: The higher the contingency level, the larger amount of contingency needed. For example, in the figure above, we are reporting the total cost of the project. Here we can observe that we are showing the 85th percentile that corresponds to a total cost of $7.8M (right delimiter).  We can say that there is only a 15% chance that we will exceed $7.8M, or alternatively, we have an 85% chance that the total cost will be less than or equal to $7.8M.  In the same figure we can also see that the 90th percentile of the total project cost is $8.02M.  We can say then that in order to increase our confidence level from 85% to 90%, we will need to add $220,000 to the total cost.

The calculation of the contingency is then accomplished by using the base cost estimate (BE) before the risk analysis was implemented, and the expected cost (EC) of the simulated results.

Some practitioners separate the contingency into two components: engineering allowance, and management contingency.

Engineering allowance (EA) is the difference between the expected cost and the base estimate:

EA = EC – BE

Management contingency (MC) is calculated using the difference between the cost at certain confidence lever (Cp) and the base estimate:

MC = Cp – EC

In our example, our BE = $6.5M; therefore, engineering allowance EA = EC – BE = 6.86M – 6.5M = $0.36M. 

For the calculation of management contingency, we use a confidence level of 85% so Cp(85%) = $7.8M; therefore, MC = Cp – EC = 7.80M – 6.86M = $0.94M.

In many situations, the suggested contingency might be excessive, so the need for a mitigation study is necessary. We can use the sensitivity analysis tool in @RISK to detect the key drivers affecting our total cost. This is valuable information so that we can concentrate our efforts in reducing the impact of risk events and uncertainties to the total cost. Below, we see a tornado graph with the most important drivers. The analyst will then explore the appropriate mitigation strategies and assess their implementation cost. A second simulation can be run to assess the effectiveness of the proposed mitigation plan, and compare the pre-mitigated and post-mitigated cost distributions.




In following blog posts, I will explain how to distribute the assessed contingency to cost elements and identified risk events in project risk management models.

Javier Ordóñez, Ph.D
Director of Custom Solutions

Simulating the U.S. Economy: Where will we be in 100 years?

Friday, September 4, 2009 by DMUU Training Team
William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics).

Dr. Strauss will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for his case study below, and see the full schedule for the Conference here.

Simulating the U.S. Economy:
Where will we be in 100 years?


There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

To see where this growth imperative will take us we first have to see how we go to where we are today. This work first models the 20th century. The model is both complex and simple. The basic schematic of the model’s relationships is easy to understand. Furthermore, the core of the model is a simple production function that combines capital, labor, and the useful work derived from energy to generate the output of the economy. Complexity is contained in the solutions to the internal workings of the model. What is unique is that there are no exogenous economic variables. Once the equations’ parameters are calibrated, setting the key outputs to "one" in 1900 results in their time paths very closely predicting the U.S. GDP and its key components from 1900 to 2006. 

The experiment in this work is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? From 1900 to 2006 there are periods in which there was parameter switching. (The optimal parameters and the years for the switching were found using a constrained optimization technique.) That suggests that in the future there will also be changes. The experiment uses @RISK’s features to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The "doing what you did gets you what you got" scenario leads to a surprising and unsettling outcome. The experiments using @RISK do find a path that works. Obviously if it is not "business-as-usual" that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

Please join us in October in New York for software training in best practicies in quantitiative risk analysis and decision making under uncertainty, real world case studies from risk services consultants and experts, and networking with practicioners from many different fields including oil and gas, pharmaceuticals, academics, finances, Six Sigma, and more.

Integrated Project Risk Analysis

Wednesday, August 26, 2009 by DMUU Training Team
2009 Palisade Conference in New York City Jay O’Connor is a Director at Turner & Townsend Inc. With over 25 years of experience in the areas of estimating, planning and quantitative risk analysis for international projects, Jay understands the complexities that are associated with identifying and assessing project risks. His experience includes both the owner’s and contractor’s side of engineering and construction projects. He has worked in the upstream and downstream oil and gas industry sectors and the pulp and paper sector. His career has taken him to the United Kingdom, Japan, Indonesia, Malaysia, Singapore and Australia.

Jay will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for his case study below, and see the full schedule for the Conference here.

Integrated Project Risk Analysis

When conducting project risk analysis, it is not uncommon for the qualitative risk, quantitative schedule and quantitative cost risk analysis to be conducted separately and kept independent of each other. While some software packages attempt to integrate all three into one analysis, these efforts tend to fall short in one area or another. Turner & Townsend’s approach is to integrate the residual risks and opportunities along with the results from the schedule risk analysis into the cost risk analysis to develop a more fully integrated project risk analysis. The presentation will discuss our approach to risk analysis.

October 21-22 in NYC

Building on the success of last year’s record-breaking event, the conference will offer a wide range of software training, model building, and real-world case study sessions. Last year, the event drew over 150 practitioners and decision-makers from a broad spectrum of industries. The @RISK and DecisionTools software tracks were more popular than ever. This year, we’re expanding software training with sessions that let you walk through examples and try the tools directly. This will enable you to take some new tips back to the office. Please join us in October for a great opportunity to learn and connect with colleagues.


Capitalizing Upon Market Inequities: A Game Plan for Successful Sports Wagering

Thursday, August 20, 2009 by DMUU Training Team
Clayton Graham is an adjunct professor of Statistics and Economics at DePaul University. He holds senior positions with the Chaos Group, Inc. and Analytical Advantages, LLC where he functions as a management consultant specializing in analytical and graphic econometrics.

He will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for Clay Graham's case study below, and see the full schedule for the Conference here.

Capitalizing Upon Market Inequities:
A Game Plan for Successful Sports Wagering


Sports wagering brings two separate "markets" together. First is the production market or the game itself.  The second is the wagering or betting market. As a matter of practicality, the wagering market is itself in balance, i.e., bet clearing is covered through the process of adjusting the cost-payout ratio (the line). Betting lines are translated into an expected probability of winning. This resultant probability is frequently inconsistent with the probability of the team actually winning. 

Hence, the opportunity to capitalize upon the dichotomy between the inequities of the production and gaming markets will be detailed and quantified. The presentation will include:
  • Fundamentals of gambling lines and odds,
  • Identification of key metrics,
  • Methods of production modeling baseball and basketball
        (similarities and differences),
  • Integration of economics (investment) with production,
  • Economics of decision making.
Principal Palisade software utilized includes: StatTools (statistical analysis toolkit for Microsoft Excel), @RISK (risk analysis software using Monte Carlo techniques in Excel) and Evolver (genetic algorithm optimization in Excel). The presentation will have a heavy graphic and visualization emphasis. Theoretical statistics will be tightly tied with pragmatic realities of game modeling and economically based decision making. 

Specific quantification will consist of:
  • Probabilities of winning a game,
  • Measurement bias of officials,
  • Quantification of player performance,
  • Expected values of return on investment,
  • Sports gambling optimizing algorithm.
Examples of actual results for current and past seasons along with predictions will be provided. 

In short, it’s "Card Counting" for sports!

More information about this project can be seen at Baseballwon.com.

Putting Monte Carlo Software in Reverse

Friday, June 26, 2009 by Holly Bailey
Question for today: What do you get when you run Monte Carlo software back in time? 
 
Answer: You get closer and closer to the wreckage of Air France Light 447.
 
The U.S. Coast Guard's search for the crash site of the doomed Air France plane was the first major test of its "reverse-drift" modeling  program SAROPS (Search and Rescue Optimal Planning System).  Earlier this year I reported on one of its first reality tests, the search for two football players whose boat capsized in the Gulf of Mexico, which apparently took place before the software was formally adopted by the Coast Guard.  For this search, a Coast Guard team in Portsmouth, Virginia, managed the modeling in close cooperation with French and Braziian rescue teams.
 
At last report, the reverse risk analysis was performing admirably.  Starting with the location of the first object sighted in the water, in this case a seat cushion and some smaller debris, team using SAROPS established the location and the immediate wind and current conditions  and then used the history of weather and water since the plane disappeared to estimate thousands of possible paths the seat cushion could have traveled to reach its location.  When the next piece of debris surfaced, its data were fed into the program, and the Monte Carlo software spun out a slightly narrower range of retrospectively possible routes.  
 
Although the reconstruction of the crash location sounds laborious, the simulations are extremely fast.  The software can spin out ten thousand possible routes in fifteen minutes, and as the possible routes of a number of objects begin to converge, they focus with increased probability on the crash site.
 
 A picture--always worth a thousand words--of this clustering of simulated pathways can be found on the Virginian-Pilot website.  If you take a clook at that online graphic, you can see how, in the case of SAROPs, hindsight gets close to twenty-twenty . 

Rethinking Monte Carlo Simulation for Retirement Planning

Friday, June 12, 2009 by DMUU Training Team
A recent article entitled “When Monte Carlo analysis meets a black swan” in Investment News addresses the criticisms Monte Carlo simulation has received for “missing the meltdown.”  The author, Moshe A. Milevsky , notes that people typically seek a single number “answer” from a Monte Carlo simulation, such as the probability of meeting a single retirement planning goal.  Milevsky points out that many Monte Carlo software packages do not include sensitivity or scenario analyses to drill down and determine which variables are really driving the risk inherent in the results.  He proposes what amounts to a stress test – simulating what could happen under likely scenarios, and simulating again under 1-in-100 chance “black swan” disastrous scenarios.  Milevsky wraps up by saying, “Instead of condemning Monte Carlo analyses for missing the meltdown, let's properly harness the full power of stochastic methods to give us tools that provide clear utility.”

I believe Milevsky makes a great point, focusing on the modeling practices rather than the tools themselves in this case.  Monte Carlo simulation tools are very important for applications like retirement planning, but even the best hammer can’t help an unskilled carpenter.  Tools like @RISK include sensitivity and scenario analysis, enabling easy implementation of tests under different scenarios for portfolio value, inflation, longevity, or any combination of these.

Randy Heffernan
Vice President

Don’t Blame the Math

Thursday, June 4, 2009 by DMUU Training Team
A recent article in Bank Investment Consultant criticized the risk analysis method of Monte Carlo simulation for not taking into account extreme events like the stock market crash. According the article, a Morningstar executive states that the “bell-shaped curve that Monte Carlo simulations use” artificially assigns the probability of what happened as zero.  Furthermore, the Retirement Income Industry Association calls for an update to Monte Carlo software simulators to include “a larger number of scenarios that assume greater volatility.”

These arguments demonstrate a fundamental lack of understanding of what Monte Carlo simulation is. The underlying assumption that Monte Carlo simulation itself is somehow to blame fails to recognize that Monte Carlo simulation is simply a mathematical technique that takes into account many different possible scenarios – but only within boundaries set by the user.  You can’t change the underlying math behind  these “what-if” calculations.

When modelers set up retirement planning or financial models, the people doing the modeling must make assumptions about the likelihood of different things happening – like the market crashing, for example.  People may make those assumptions based on historical evidence or expert opinion, but it’s people who make those assumptions – not Monte Carlo simulation software.  People then enter their assumptions into a Monte Carlo simulation model, setting up probability distributions to reflect their chosen likelihoods of occurrence.  If the assumed volatility is insufficient, that is the fault of the modelers, not the simulation itself.

In addition, Monte Carlo simulation does not always “use” a bell-shaped curve.  Uncertainty can be modeled with dozens of different probability distributions, many of them not bell-shaped.  And the call to include more scenarios and volatility can easily be met by existing Monte Carlo software such as @RISK. It’s simply up the user to change the model parameters to look at more possible outcomes. The Monte Carlo simulation package won’t fight it.

It’s disappointing to see esteemed financial organizations such as Morningstar and the Retirement Income Association missing the point. Calling for changes to Monte Carlo simulation itself is not only impossible but fails to recognize the problems with modeling practices that led everyone to miss the crash. 

Randy Heffernan
Vice President

Fatter Tails

Thursday, May 7, 2009 by Holly Bailey
At the height of panic--and consternation--over turmoil in the financial sector at the turn of the year, many an accusatory finger was pointed at the risk analysis models the finance industry used to establish the value of various types of debts. Often the particular charge was that the simulations produced by Monte Carlo software lacked not only precision but even the capacity for precision. At the time, I responded in this blog that a risk assessment model is only as reliable as the probabilities it is build on.  
 
Now some folks in financial planning firms whose customers experienced the unhappy results of models created with their companies' proprietary Monte Carlo software are becoming believers.  They are revisiting the probability distributions that generated those risk simulations, and in comments to the press are citing the need for distributions with fatter tails in order to account for randomness over longer time periods, in order to foresee a Black Swan event. This is no doubt due to the sudden prominence of author Nasim Nicholas Taleb, whose recent fame is due to his timely second-guessing of the markets.  
 
That people who work in finance and investing should start listening to a critic only after coming to grief is not surprising. What I do find surprising, however, is the planners who mention to reporters that they have been limited to the standard bell curve distribution or that their software doesn't provide for sensitivity analyses. Any number of commercially available Monte Carlo software packages have been offering a fairly wide choice of distributions, along with sensitivity analyses for quite a while. Fatter tails aren't hard to come by, so why can't planners seem to find them?

The Trials of Trials

Friday, May 1, 2009 by Holly Bailey
In my last blog I mentioned there has been a dramatic upswing in the use of risk analysis and Monte Carlo software in clinical trials for new drugs.  A new unpublished paper by Todd Clark of VOI Consulting makes clear some of the reasons more people in the pharmaceutical industry are turning to operational risk software to guide them in setting up trials.   
 
First of all, a clinical trial is probably not one trial but a process involving a series of trials, each of which takes a number of years and millions of dollars to complete.  This process takes place before the company even presents the drug to the FDA for approval.  Then, as the U.S. Government Accountability Office, points out, the FDA eventually approves only 1 in 10,000 compounds a safe and effective.  No wonder--again according to the GAO--"the number of new drugs being produced has generally declined while research and development expenses have been steadily increasing."
 
Although there are enormous profits to be made if a drug developed for a large number of patients is approved, there are great sums of money to be lost and many tricky decisions to be evaluated along the way to successful product strategies.  As Clark points out, even the planning of a single clinical trial is itself fraught with uncertainty: How many subjects?  What kind of subjects?  What kinds of physicians?  Where to hold the trials?  And the answer to each of these questions is in turn a balancing out of numerous variables.
 
So there's plenty of risk to go around.  But potentially plenty of reward.  Just made for risk assessment with Monte Carlo.

Modeling in Process Development - Deterministic or Stochastic?

Tuesday, April 21, 2009 by Steve Hunt
Deterministic" and "stochastic" sound like fancy, academic words, but they are vital for everyone in business to understand. Deterministic refers to single numbers, while stochastic refers to probabilities. When modeling a system, a deterministic model uses numerical input values and calculates numerical output values. But a stochastic model uses a probability distribution for each input and estimates a probability distribution for each output. Monte Carlo simulation (@RISK) is the standard tool for stochastic modeling. During Monte Carlo simulation, random values are generated for inputs, and output values are calculated. By repeating this process many times and saving output values, the simulation generates estimates for output distributions.
 
Stochastic modeling and Monte Carlo simulation have become accepted tools in financial professions. However, engineers working on the development of new products have been slow to adopt these important tools, opting instead for deterministic approaches such as worst-case analysis. These deterministic tools answer some questions, but they are unable to predict process capability, which is the key to success in a Six Sigma environment. Monte Carlo simulation is now an essential tool for every engineer involved in product and process development. Stochastic tools can predict and prevent performance and capability problems before the first prototype is built. What once required months or years to discover now takes only minutes to prevent.
 
On April 30th, Andrew Sleeper  of Successful Statistics will be presenting  Accelerating Product Design with Simulation and Stochastic Optimization. Plan to spend hour to learn how to save months in your product and process development projects. It will be time well spent. You can register to attend at www.palisade.com/seminars/webcasts.asp

Social Influence and Network Exploitation

Sunday, April 19, 2009 by Holly Bailey
It was bound to happen.  Online communities such as FaceBook and Twitter, which are themselves commercial animals, are being mined by all kinds of enterprises from ad agencies to credit card companies for the commercially valuable data they can yield.  A recent opinion piece in the Manchester Evening News rounds up a fair number of potential uses of this socially generated data and tries to sort out the good from the not so good, and the bad from the truly ugly.
 
According to the Evening News's Paul Taylor, businesses are using social networking sites for everything from checking out individual job applicants to statistical analysis of myriad purchasing decisions with neural network technology.  On of the worrisome scenarios he highlights is the probable effect of upcoming legislation by Parliament that would require law enforcement agencies to keep records of web traffic.  Another is the move by Google to obtain customers' permission to let Google use cell phone software to keep track of their whereabouts and apply its operation research magic to turn the information it acquires this way into marketable fact.  But he balances these possibilities with other brighter ones--such as helping doctors do better risk assessment in creating treatment plans.
 
Falling somewhere in the space between sinister and beneficial is the use of social networking data for marketing.  About the same time as Paul Taylor's opinion piece was published, a marketer's blog for the auto industry laid out the conceptual framework of a strategy based on online communities that it has trademarked as "Social Influence Marketing."  A component of any campaign as essential, it claims, as direct marketing and branding.
 
At the moment, all of this should mean more to you if you are young, because, at the moment, the young are the people who are most attracted to social networks.   And they are the ones who will immediately see the utility of network data for marketing and product strategies.  But if, as they say, youth is only a state of mind, it won't be long before the rest of us catch up and catch on as the social network and its exploitation evolve.  

Is Norway’s Pension Fund Adequately Diversified? Part II

Tuesday, April 7, 2009 by DMUU Training Team
In an earlier blog I allowed myself some raw speculation as to whether holistic risk management thinking is being adequately applied when it comes to the Norwegian government’s management of the state pension fund. This fund represents one of the world’s largest exercises in risk analysis in “retirement planning.” The Fund invests the oil wealth generated in the country in a mix of global equities and % bonds, and whose performance is essentially currently measured against a global benchmark portfolio of bonds and equities.

I specifically asked the question as to whether the fund should be devoting far more significant efforts to invest in non-traditional assets, as a way to mitigate some potential scenarios that could adversely affect both equity and bond investments. Investments that could potentially mitigate some of these scenarios could perhaps include very large positions in alternative energy technologies, and I noted however that although the costs and risks of such an investment strategy could be large (particularly as the scenario which it mitigates may never materialize), it could nevertheless be a prudent one, given the already very large fund that already exists for a small population base of about 5m people. Could it be so bad if 5%-10% of that fund were invested in such technologies (with such investments arguably supporting some of the fund’s other goals – such as ethical or social investments)?

It is therefore with interest that the Financial Times reported last Saturday  that the Norwegian government is planning to review the operations of its sovereign wealth fund after it lost €75bn ($100bn, £68bn) on investments last year. I await eagerly the results of this review, specifically of course to see whether such fundamentally new investment strategies will be implemented.


Dr. Michael Rees
Director of Training and Consulting

Best Practices in Risk Modelling

Wednesday, April 1, 2009 by DMUU Training Team
The recent blog positing on best practices in Excel modelling could be thought of as providing a reasonable and robust set of principles for building static Excel models. When building simulation models for risk analysis in Excel (for instance, with @RISK Monte Carlo software), some other points are worthy of consideration:
  • A risk model may need to be built at an appropriate level of detail. A model which is too detailed will be more complex to add risk distributions to and will require more effort to capture the dependencies between the larges number of variables. In many practical cases, key dependencies will simply not be captured, and the result will have an excessively narrow range (for additive type models e.g. cost budgeting) or an excessively wide range (for subtractive models e.g. profit as the difference between uncertain revenues and costs).
  • The inclusion or not of event risks. Generically, a static model of a situation in which there are event risks (e.g. something adverse happening in 20% of cases in a reserve estimation model) would not include such a risk as a line item (since the most likely outcome is its non-occurrence), whereas a risk model would.
  • The prioritisation of event risks to include may be non trivial, and depend on the decision maker's risk profile (i.e. tolerance and decision-making criteria), as well as the potential total number of event risks under consideration.  For example, in a retirement planning model where a decision is to be based on the P90 outcome (i.e. the worst or best 10% of cases) it would be more important to include an event with an impact of 1 (with 100% probability) than an event with impact of 100 with 1% probability, as this latter event in isolation would have very little effect on the P90 of the output distribution (Were decisions to be made on the P99.5 value, we would have a different situation, of course.).
  • The use of DataTables will generally slow down simulation models, as the tables need to be recalculated at every iteration. DataTables may be used when building to model and as an error-checking tool (TopRank may also be used, to check that errors are zero across a range of scenarios), but may need to be removed before running the final simulation model.
  • The real challenges in risk modelling boil down to those related to model formulation and decision-making; that is: the selection of variables, capturing the true dynamic of the situation in the model, choice of distributions, capturing dependencies correctly etc. (so come to a training course!)
  • Other principles of model implementation (as discussed in the earlier blog) are essentially identical.

Dr. Michael Rees
Director of Training and Consulting

Real World Gamesmanship

Monday, March 30, 2009 by Holly Bailey
A few years ago, the online magazine for computer gaming Gamesutra published an article by a game developer Alan Carpenter extolling the virtues of using risk analysis to balance role-playing games.  In this case balance meant designing a game that was neither too difficult nor too easy, and it was the costly, time-consuming goal of game development companies--costly meaning an average of $3 million a title!

Carpenter had observed that by using the same operational risk software that the oil and gas industry uses to make decisions under uncertainty, a game designer could take advantage of the thousands of possible scenarios spun out by a Monte Carlo simulation to add a new element of reality to games where context and emotion may be big draws for the gamer but can't sustain entertaining play.

Carpenter advises game developers that many games could be designed using the same Monte Carlo Excel spreadsheet.  He offers a lengthy technical blueprint for games that are based on conflict--war, street fights, etc.--and in revisiting it what intrigues me is the idea that the same infrastructure of algorithms and probability functions that Carpenter lays out to capture events in an imaginary world could just as easily be applied to real-world political and military events.

If risk simulation is being used to help plan world events, it's not widely talked about.  But I suspect this is going on, and I would love to hear from anyone out there who knows more about this than I do.  

Advanced Analytics for Business Intelligence

Thursday, March 12, 2009 by DMUU Training Team
Business Intelligence (BI) is all the rage. Businesses want business intelligence. Analytics and reports are at the heart of BI. Decision makers want decision intelligence. Analysis, especially quantitative risk analysis and Monte Carlo simulation, yields more thorough intelligence for effective decision making under uncertainty.

Some, Ralph Kimball among them, challenge that advanced analytical tools, “as powerful as they are, can be understood and used effectively only by a small percentage of the potential … business-user population.” What’s missing from the assertion is a recognition that data are about what has already happened. If you’re forecasting or planning strategically, you need predictions moving forward. It’s not just about data mining, it’s how you employ the data to make effective and well-informed decisions.
According to one BI developer, “use of advanced analytics has been limited to power analysts” leaving reporting capabilities to the bottom of the technology pyramid. But risk and decision analysis tools from Palisade Corporation are not just for power analysts — anyone can make ready use of these Monte Carlo software tools. Besides, several large corporations make @RISK accessible to business and production sides in the organization.

@RISK, and each of the DecisionTools Suite applications, has built-in reporting capabilities to communicate the analyses you’ve performed. In @RISK 5, you can even have your reports generated immediately after completing a simulation (see the Simulation Settings dialog), whether those reports are in standard form and layout, or as customized templates. Any statistic you can find from a data set can be reported in easy to use Excel spreadsheets. The quants can utilize the reports to communicate the risk analytics of a strategic decision as well as those on the front lines who need to generate a quick assessment of uncertainty in operational actions.




Thompson Terry
Palisade Training Team

Recalculating the Calculators

Thursday, March 12, 2009 by Holly Bailey
In an article this week in the online publication TheStreet, Taylor Smith takes aim at online calculators for retirement planning.  He says the problem with them is that most of them are based on Monte Carlo software, and then goes on to make some broad and inaccurate statements about the characteristics about Monte Carlo simulation and the ways it skews risk assessment.  

For starters, he quotes an investment "expert," who is, coincidentally, president of concern that produces risk analysis software not driven by Monte Carlo software, to the effect that good financial planning should take into account, more than market returns--including taxes, income, and expenses.  This implies the Monte Carlo simulation is not capable of factoring these elements into its predictions.  Nothing, as the a recent risk analysis model offered by both the Society of Actuaries and the Casualty Society aptly demonstrates, could be farther from the truth.  This model, developed by professors of finance and mathematics at Illinois State University and the University of Illinois Champaign-Urbana, helps pension and insurance planners to to forecast not only projected market returns but the effects such critical economic factors as interest rates, equity price levels, inflation and unemployment rates, and real estate prices.

My point is that a Monte Carlo simulation is what you make it.  It can be very simple and limited to one or two economic factors, or it can be a complex mix of many factors.  If the online retirement calculators are too simplistic to usefully account for reality, their builders have plenty of room to improve them.     

Lost at Sea

Monday, March 9, 2009 by Holly Bailey
Last week, when the U.S. Coast Guard called off its search for three men, including two NFL football players, who were pitched into the Gulf of Mexico when their fishing boat capsized, its spokesman commented, "We're extremely confident that if there are any survivors on the surface of the water that we would have found them."

Following up on this, Scientific American interviewed an oceanographer in the Coast Guard's Office Search and Rescue, to ask about the source of this confidence.  It turns out that the Coast Guard uses Monte Carlo software to plan  every search operation.  It's the basis of the system they call SAROPS (Search and Rescue Optimal Planning System).  What SAROPS simulates is the drift trajectory for search targets--people, vessels.  Based on historical data on drift patterns for similar objects, it projects drift scenarios for various starting locations and times.  At the same time, it factors in an environmental risk analysis of wind and current.

The models produce tightly defined options for search patterns, which save search time and increase the likelihood of finding castaways while they are still alive.  But, as the Coast Guard oceanographer pointed out about the case of the missing fishermen, "There’s always uncertainty, of course, which is why we’re having a search in the first place."

A Good Read on a Bad Year

Monday, March 2, 2009 by Holly Bailey
Warren Buffet did not have a good year.  This fact about 2008 was detailed last week in a New York Times article and in a letter from Mr. Buffet to stockholders of his Berkshire Hathaway company.  This letter offers plenty of plain-spoken criticism of the risk assessment failures in the financial sector to go around--'beware of geeks bearing formulas." 

But he reserves his harshest criticisms for derivatives--"weapons of mass destruction.".  He not only cites the hazards of using risk analysis models to value these complexly structured investment products, but he points out something I haven't seen mentioned before: derivatives create a "web of mutual dependence" among financial institutions that lingers for years.

About this web of dependence, he says, “Participants seeking to dodge troubles face the same problem as someone seeking to avoid venereal disease,” he wrote. “It’s not just whom you sleep with, but also whom they are sleeping with.”

I recommend Warren Buffet's letter if only for its Olympian view of the exponential decline in the credit markets and his list of the intriguing entertainments he has planned for his shareholders at their annual meeting this May.  His bad year makes for some good reading.