Monte Carlo simulation

@RISK Creates an 80% Cost-Savings for Nuclear Power Company

@RISK Creates an 80% Cost-Savings for Nuclear Power Company Building a new power plant in the U.S. is not for the risk-averse. The average cost of building one of these plants is historically 300% of the planned cost at the start of construction. Because of these prohibitive cost risks, as well as unfavorable public sentiment in reaction to the Three-Mile Island accident in 1979, U.S. nuclear power plant construction had ground to a halt for decades.

Almost 30 years later, the first contracts for new plants were signed by a major U.S. nuclear power company in 2008. The total expected project cost for the three projects was over $30 billion. Hoping to avoid the abysmal track record of cost overruns in the industry, the nuclear power company wanted to use a new and improved risk-management approach to the planning process—so they hired Dr. Sola Talabi, Risk Management Professor at Carnegie Mellon University and Risk Management Consultant for Pittsburgh Technical, to serve as Nuclear Power Plant Risk Manager. “They knew what the history had been with these projects, and they didn’t want it to happen again,” says Dr. Talabi.

In previous power plant construction projects, risk management was only an objective of project managers. This time, risk management was both an objective and a function; the company created a dedicated risk organization whose only role was to manage risk. Furthermore, the company switched from deterministic risk planning methods, to probabilistic methods—using @RISK. Armed with the software, Dr. Talabi and his risk management team were able to characterize the uncertainty associated with cost items and perform quantitative risk analysis using Monte Carlo simulation. They were also able perform a sensitivity analysis to identify key cost drivers, such as the fact that supply chain issues was the greatest risk driver of all.

Thanks to this analytical, @RISK-driven approach, Dr. Talabi and the risk organization were able to dramatically impact these nuclear power plant projects, reducing cost overruns by roughly 80% relative to the historical trend. “While we still have cost overruns, it’s nothing like what we had before,” says Dr. Talabi. “I think it’s very important for decision-makers to realize that the rampant cost overruns of the past can be much better managed using tools like @RISK.”

» Read the complete case study here.

 

@RISK Helps Zero-In on U.S. Senate Race Outcomes

The midterm Senate race is fast approaching—and so are the speculations on its outcome. Previously, Lawrence W. Robinson, Professor of Operations Management at Cornell University’s Johnson Graduate School of Management used @RISK to statistically predict the senate races, using data from the stats-centered news site, FiveThirtyEight.   FiveThirtyEight was founded by statistician and political analyst Nate Silver, who, in his forecasts earlier in the year, initially summed up the probabilities of either Democrats or Republicans winning all their races.

Robinson took this analysis a step further by adding Monte Carlo simulation to the mix. While Silver warned in previous articles that to assume races are uncorrelated is “dubious,” and that Monte Carlo simulations requires variables to be uncorrelated, Robinson demonstrated that it is in fact very possible to include correlation in Monte Carlo analyses.

He started by creating a “lower bound” (zero correlation) and an “upper bound” (total correlation) in his model, and showed that Democrats’ chances of retaining control only fell somewhere between 41% and 50%.

Robinson_Probability

The FiveThirtyEight Approach

Fast-forward a few months, and FiveThirtyEight’s models have gotten considerably more complex and data-rich, and their interactive forecasts are updated almost daily. As of this writing, the model predicts that Democrats have a 42% chance of retaining the Senate next year.

538_2014Senate_Probabilities

Unlike their earlier forecasts, “they’ve also included a correlation, of a type in their model,” says Robinson. “They do not explicitly use a correlation coefficient, as I did—instead, they change the distribution of the candidate’s lead.” Robinson explains that Silver and FiveThirtyEight introduce correlation through an additional random variable representing what they’ve labelled “national error,” which they generate and add to the mean margin of victory of every candidate.

This national error “could be a sex scandal, or some underlying and largely uncaptured sentiment in the nation,” Robinson explains. “For example, in the 2012 presidential race, it might have been Hurricane Sandy, and how presidential Obama looked in response.”

In the FiveThirtyEight forecast model, if the national error (whatever it represents) turns out to be +3 for the Republicans, they shift the mean margin of victory three points towards each and every Republican. “Unfortunately, nowhere in their post do they specify the probability distribution for his new ‘national error’ random variable,” Robinson says. “Thus it is not possible to know how correlated the individual races are with one another.”

@RISK Presents an Alternative Method

Because FiveThirtyEight’s methods are not entirely clear, Robinson wanted to devise a way to arrive at these forecasts using his own statistical methods, and to use a correlation that is explicitly defined. Instead of just using FiveThirtyEights’s “Leader’s chance of winning,” which was only given to the nearest percent, Robinson started with the mean and (estimated) standard deviation of the margin of victory, and calculated the probability of winning by assuming that the margin of victory on Election Night was normally distributed. “Although Silver says he assumes that the victory margin is leptokutic [has fat tails] for finding the probability of winning, he never specifies its probability distribution,” says Robinson. “I found that the standard assumption that the margin was normally distributed better matched his reported analysis.”

Robinson then built a Monte Carlo model in Excel using @RISK, treating the outcome of each race as a Bernoulli (0/1, win/lose) random variable. He then introduced a correlation matrix that captured the correlation between every pair of races, and ran 27 different simulations (each one simulating 400,000 elections) for correlations ranging between 0% and 100%. His results closely match that of FiveThirtyEight’s, showing the probability that Democrats will retain control of the Senate as a function of the correlation among races. “Now we can say that, as long as the correlation is between 20% and 97%, the probability that the Democrats will retain control will be between 40% and 42%,” says Robinson. “The advantage of this approach,” Robinson says, “is that we specify the correlation precisely, and that we conduct robustness analysis to see how the results change with the correlation.”

Interested in playing a political prognosticator? Check out our models here and run the @RISK simulation yourself.

 

@RISK Takes Nate Silver’s Senate-Race Predictions a Step Further

JohnsonSchool

Statistician super-star and  FiveThirtyEight editor Nate Silver has gained fame for his spot-on predictions around elections; he had accurate predictions for all 50 states during the 2012 presidential election. While it seems that Silver has the power of precognition, he actually relies on refined statistical methods—including weighting various political polls—to make predictive models.

Recently, Silver and FiveThirtyEight put out a forecast for the 2014 Senate race which examines the races on a probabilistic basis. Silver’s analysis, which simply sums the probabilities of each side winning all its races, projects that the Democrats are slightly more likely to lose control of the chamber than to retain it.

Lawrence W. Robinson, Professor of Operations Management at Cornell University’s Johnson Graduate School of Management took this research one step further by adding Monte Carlo simulation to the mix. Robinson set out to determine, in his words, “the probability that the Democrats hold at least 50 seats in the new Senate.” Only 50 seats are needed because Joe Biden will, in his role as president pro tempore of the Senate, break ties in the Democrats’ favor. “What we really want to know is, what chance will the Democrats have to retain control”?

After using @RISK to crunch the numbers, Robinson found that the Democrats have only a 41% chance of retaining control of the Senate.

While Silver warned in previous articles that to assume races are uncorrelated is “dubious,” and that Monte Carlo simulations requires variables to be uncorrelated, Robinson demonstrated that it’s in fact very possible to include correlation in Monte Carlo analyses.

By creating a “lower bound” (zero correlation) and an “upper bound” (total correlation) Robinson showed that Democrats’ chances of retaining control only hovers somewhere between the aforementioned 41% and 50%.

With the upper and lower bounds  in place, Robinson went on to create a model that allows the coefficient of correlation between every pair of elections to vary between 0% and 100%, and found the probability that the Democrats will hold the Senate for each different correlation coefficient value.natesilver1

As Robinson says, “It would be very difficult to determine the correlations among all the different Senate races. However, if the coefficient of correlation is anywhere in the wide range between 20% and 85%, then the probability that the Democrats will retain control of the Senate (i.e., hold 50 or more seats) will be in somewhere in the very narrow band of 45% ± 0.1%.”

Stay tuned as the mid-term Senate race approaches this fall—Robinson plans to model more nuanced and data-rich predictions as the election nears.

Palisade’s Impact on Helicopter Company Featured in Widespread Press

BloombergBusinessweek-Logo-940x350Palisade continues to garner broad media attention, this time due to the use of our software by a major helicopter manufacturer looking to better analyze risk when introducing new products. AgustaWestland, the European helicopter manufacturer, turned to @RISK to determine the financial feasibility of developing any new product, as detailed in our recent case study and blog. Since then, other publications have picked up the story, including Global Banking & Finance Review, a leading portal for financial news and opinions, Helihub, a data source for the helicopter industry,  Bloomberg Business Week, IT News Online, and Gadget Rumours. The story features a Palisade case study on the helicopter manufacturer’s efforts to conduct  risk analysis using  Monte Carlo simulation to determine the accuracy of their forecasts, and in turn improve business, both in true feasibility and in financial results.