Nate Silver

@RISK Helps Zero-In on U.S. Senate Race Outcomes

The midterm Senate race is fast approaching—and so are the speculations on its outcome. Previously, Lawrence W. Robinson, Professor of Operations Management at Cornell University’s Johnson Graduate School of Management used @RISK to statistically predict the senate races, using data from the stats-centered news site, FiveThirtyEight.   FiveThirtyEight was founded by statistician and political analyst Nate Silver, who, in his forecasts earlier in the year, initially summed up the probabilities of either Democrats or Republicans winning all their races.

Robinson took this analysis a step further by adding Monte Carlo simulation to the mix. While Silver warned in previous articles that to assume races are uncorrelated is “dubious,” and that Monte Carlo simulations requires variables to be uncorrelated, Robinson demonstrated that it is in fact very possible to include correlation in Monte Carlo analyses.

He started by creating a “lower bound” (zero correlation) and an “upper bound” (total correlation) in his model, and showed that Democrats’ chances of retaining control only fell somewhere between 41% and 50%.

Robinson_Probability

The FiveThirtyEight Approach

Fast-forward a few months, and FiveThirtyEight’s models have gotten considerably more complex and data-rich, and their interactive forecasts are updated almost daily. As of this writing, the model predicts that Democrats have a 42% chance of retaining the Senate next year.

538_2014Senate_Probabilities

Unlike their earlier forecasts, “they’ve also included a correlation, of a type in their model,” says Robinson. “They do not explicitly use a correlation coefficient, as I did—instead, they change the distribution of the candidate’s lead.” Robinson explains that Silver and FiveThirtyEight introduce correlation through an additional random variable representing what they’ve labelled “national error,” which they generate and add to the mean margin of victory of every candidate.

This national error “could be a sex scandal, or some underlying and largely uncaptured sentiment in the nation,” Robinson explains. “For example, in the 2012 presidential race, it might have been Hurricane Sandy, and how presidential Obama looked in response.”

In the FiveThirtyEight forecast model, if the national error (whatever it represents) turns out to be +3 for the Republicans, they shift the mean margin of victory three points towards each and every Republican. “Unfortunately, nowhere in their post do they specify the probability distribution for his new ‘national error’ random variable,” Robinson says. “Thus it is not possible to know how correlated the individual races are with one another.”

@RISK Presents an Alternative Method

Because FiveThirtyEight’s methods are not entirely clear, Robinson wanted to devise a way to arrive at these forecasts using his own statistical methods, and to use a correlation that is explicitly defined. Instead of just using FiveThirtyEights’s “Leader’s chance of winning,” which was only given to the nearest percent, Robinson started with the mean and (estimated) standard deviation of the margin of victory, and calculated the probability of winning by assuming that the margin of victory on Election Night was normally distributed. “Although Silver says he assumes that the victory margin is leptokutic [has fat tails] for finding the probability of winning, he never specifies its probability distribution,” says Robinson. “I found that the standard assumption that the margin was normally distributed better matched his reported analysis.”

Robinson then built a Monte Carlo model in Excel using @RISK, treating the outcome of each race as a Bernoulli (0/1, win/lose) random variable. He then introduced a correlation matrix that captured the correlation between every pair of races, and ran 27 different simulations (each one simulating 400,000 elections) for correlations ranging between 0% and 100%. His results closely match that of FiveThirtyEight’s, showing the probability that Democrats will retain control of the Senate as a function of the correlation among races. “Now we can say that, as long as the correlation is between 20% and 97%, the probability that the Democrats will retain control will be between 40% and 42%,” says Robinson. “The advantage of this approach,” Robinson says, “is that we specify the correlation precisely, and that we conduct robustness analysis to see how the results change with the correlation.”

Interested in playing a political prognosticator? Check out our models here and run the @RISK simulation yourself.

 

@RISK Takes Nate Silver’s Senate-Race Predictions a Step Further

JohnsonSchool

Statistician super-star and  FiveThirtyEight editor Nate Silver has gained fame for his spot-on predictions around elections; he had accurate predictions for all 50 states during the 2012 presidential election. While it seems that Silver has the power of precognition, he actually relies on refined statistical methods—including weighting various political polls—to make predictive models.

Recently, Silver and FiveThirtyEight put out a forecast for the 2014 Senate race which examines the races on a probabilistic basis. Silver’s analysis, which simply sums the probabilities of each side winning all its races, projects that the Democrats are slightly more likely to lose control of the chamber than to retain it.

Lawrence W. Robinson, Professor of Operations Management at Cornell University’s Johnson Graduate School of Management took this research one step further by adding Monte Carlo simulation to the mix. Robinson set out to determine, in his words, “the probability that the Democrats hold at least 50 seats in the new Senate.” Only 50 seats are needed because Joe Biden will, in his role as president pro tempore of the Senate, break ties in the Democrats’ favor. “What we really want to know is, what chance will the Democrats have to retain control”?

After using @RISK to crunch the numbers, Robinson found that the Democrats have only a 41% chance of retaining control of the Senate.

While Silver warned in previous articles that to assume races are uncorrelated is “dubious,” and that Monte Carlo simulations requires variables to be uncorrelated, Robinson demonstrated that it’s in fact very possible to include correlation in Monte Carlo analyses.

By creating a “lower bound” (zero correlation) and an “upper bound” (total correlation) Robinson showed that Democrats’ chances of retaining control only hovers somewhere between the aforementioned 41% and 50%.

With the upper and lower bounds  in place, Robinson went on to create a model that allows the coefficient of correlation between every pair of elections to vary between 0% and 100%, and found the probability that the Democrats will hold the Senate for each different correlation coefficient value.natesilver1

As Robinson says, “It would be very difficult to determine the correlations among all the different Senate races. However, if the coefficient of correlation is anywhere in the wide range between 20% and 85%, then the probability that the Democrats will retain control of the Senate (i.e., hold 50 or more seats) will be in somewhere in the very narrow band of 45% ± 0.1%.”

Stay tuned as the mid-term Senate race approaches this fall—Robinson plans to model more nuanced and data-rich predictions as the election nears.