Design and Analysis of Experiments

Fast-Moving Goods, Risk Analysis, and Spend Control

Posted Thursday, July 2, 2009 by Holly Bailey

It seems reasonable that in a recession, cost containment would become hot topic, and in the world of supply chain management, it's known as "spend control."

I was interested in a recent blog describing spend control in the "fast-moving goods" (what used to be called consumer packaged goods) industry, where taming volatility in cash flow is a huge challenge. In it, the chief procurement officer at DelMonte pointed out that forward contracting for commodities is a standard practice that involves a great deal of decision making under uncertainty--and is therefore wide open to improvement through risk analysis. Apparently this was news to many in Dave McLain's audience.

The key to profitable forward contracting, McLain observed, is to structure contracts so that your suppliers don't push their risk to your company. Not always possible, he acknowledges, but his company uses a variety of operational risk software to identify and anticipate the biggest drivers of operations risk in the commodity markets where they are active. This is a complex set of analyses, and among other techniques, he recommends using Monte Carlo software to go beyond the usual option valuation and look at production forecasting and statistical analysis of historical data on cost and price volatility--from the supplier's standpoint. The risk assessment models will churn out a range of what potential commodity costs could be, so the procurement folks can build caps and locks into their contracts.

All of this requires some analytical knowhow, and the Spendmatters blog promises a primer on the DelMonte approach. It's worth following because it ain't easy, it ain't simple, but it's how you get to spend control.

Announcing @RISK 5.5 and DecisionTools Suite 5.5!

Posted Wednesday, July 1, 2009 by DMUU Training Team

Building on the worldwide success of @RISK 5.0 and DecisionTools Suite 5.0, Palisade is pleased to announce the version 5.5 release of these best-selling risk and decision analysis tools.

New @RISK 5.5 - risk analysis with Monte Carlo simulation

Current @RISK 5.0 users will benefit from faster simulations — 2x to a remarkable 20x times faster than before — as well as new scatter plots from scenario analyses, a freehand distribution artist, and an Excel-style Insert Function dialog and graphs. @RISK 5.5 brings a range of new features to improve your analysis, save time, and encourage systematic adoption of risk analysis across your organization.

@RISK 5.5 is the best Monte Carlo simulation package available today, blending high-powered analysis with highly intuitive ease-of-use. The bottom line for you is a better understanding of what could happen and how likely it is to happen. Applications include value-at-risk, design and analysis of experiments, discounted cash flow analysis, exploration and production, option valuation, and more.

@RISK 5.5 has been fully translated into Spanish, German, French, Portuguese and Japanese.

» What's New in @RISK 5.5
» Download a free trial version

Putting Monte Carlo Software in Reverse

Posted Friday, June 26, 2009 by Holly Bailey

Question for today: What do you get when you run Monte Carlo software back in time?

Answer: You get closer and closer to the wreckage of Air France Light 447.

The U.S. Coast Guard's search for the crash site of the doomed Air France plane was the first major test of its "reverse-drift" modeling program SAROPS (Search and Rescue Optimal Planning System). Earlier this year I reported on one of its first reality tests, the search for two football players whose boat capsized in the Gulf of Mexico, which apparently took place before the software was formally adopted by the Coast Guard. For this search, a Coast Guard team in Portsmouth, Virginia, managed the modeling in close cooperation with French and Braziian rescue teams.

At last report, the reverse risk analysis was performing admirably. Starting with the location of the first object sighted in the water, in this case a seat cushion and some smaller debris, team using SAROPS established the location and the immediate wind and current conditions and then used the history of weather and water since the plane disappeared to estimate thousands of possible paths the seat cushion could have traveled to reach its location. When the next piece of debris surfaced, its data were fed into the program, and the Monte Carlo software spun out a slightly narrower range of retrospectively possible routes.

Although the reconstruction of the crash location sounds laborious, the simulations are extremely fast. The software can spin out ten thousand possible routes in fifteen minutes, and as the possible routes of a number of objects begin to converge, they focus with increased probability on the crash site.

A picture--always worth a thousand words--of this clustering of simulated pathways can be found on the Virginian-Pilot website. If you take a clook at that online graphic, you can see how, in the case of SAROPs, hindsight gets close to twenty-twenty .

The Tank, The Volkswagen, and the Neural Network

Posted Thursday, June 25, 2009 by Holly Bailey

Six years ago when Dale Addison was speaking to a group of engineers and trying to pitch "artificial intelligence"--meaning neural networks someone in his audience asked him if it was true that a neural network had once mistakenly classified a T-62 tank as a Volkswagen. Although the incident had occurred years before, Addison seems to admit that there was some truth to the story. At the time, he was dismayed to see how little confidence this technologically sophisticated audience had in neural nets because even by that time the technology had made huge progress since computer scientists started tinkering with it.

Addison is on the faculty of the University of Sunderland, and in spite of his audience's skepticism, he should be feeling fairly smug that many of the applications he foresaw for neural network technology, especially those that involve accurate classification, have been exploited. We have customers who use it on a daily basis for such tricky classification tasks as cancer diagnosis, emergency response systems, exploration and production of gas and oil, and operations research issues in manufacturing.

Nevertheless, Addison still sees resistance to emerging artifical intelligence techniques among engineers and business people, and he is still out there pitching neural nets, especially their use in combination with other new computational analysis methods, such as genetic algorithm optimization and neuro-fuzzy logic. Addison himself is working on some really tough classification problems now in the CASSANDRA project, where the goal is to develop an insider trading and market abuse detection system. He doesn't worry about the tank and the Volkswagen because he's confident that sooner rather than later, a neural network will learn to recognize a suspicious transaction when it sees one.

Risk Analysis Stress Test for Your Personal Portfolio

Posted Friday, June 19, 2009 by Holly Bailey

As anyone who has read a few of my blog entries knows, probably all too well, I believe that Monte Carlo simulation has been unfairly maligned for its role in derailing the economy. This month in his column for Seeking Alpha, Geoff Considine made this point a lot better than I've been able to make it, and he also made it more fun. Considine, who works for a firm that develops specialized Monte Carlo software for investors, offers a detailed recipe for stress testing your personal investment portfolio.

Of course, his recipe makes use of his company's software. But you could use the same recipe with the same ingredients in any Monte Carlo Excel spreadsheet (and here I don't see any need to hide my own risk analysis affiliation) and bake up the same pie charts.

What's good about his recipe is that it walks you through the assumption stages of model building quite carefully. What's fun about his recipe is that his hypothetical example, which uses a straw man named Bob, is retrospective, and he constructs a model that begins in 2007 and runs forward from there. This means Considine has an opportunity to second-guess the assumptions--such as asset value, value-at-risk, and probabilities of worst-case scenarios--that brought down the house in 2008.

Of course, mortality tables being what they are, Bob dies in the end. But Bob doesn't fail, he goes out the smart way.

Where the Wild Things Are and Risk Assessment

Posted Monday, June 15, 2009 by Holly Bailey

Over the past three years I've been tracking an uptrend in risk analysis in what might seem an unlikely field, wildlife conservation. But on further thought, this makes perfect, straightforward sense. At-risk animal populations could use some analytical help sorting out the live-or-die questions.

The first benchmark occurred when the World Conservation Union began to use Palisade's Monte Carlo software to train field biologists concerned about disease control in the WCU's projects around the world. At the time I was told that "there are hundreds of conservation projects that need to account for the risk of disease."

Then just about the time I had come to the conclusion that I could head toward the seafood department in the supermarket primed for decision making under uncertainty and confident of my choices, I became aware that an economist friend of mine who studies fish populations was running not only statistical analysis of food fish populations but using the Monte Carlo features in Excel to forecast the results of various "harvesting" scenarios on the populations of my favorite fish, the Atlantic cod.

Today, I learned that a very rare marine mammal, the Hector's Dolphin, which inhabits micro territories off New Zealand, is the latest beneficiary of environmental risk analysis. Before a Hector's Dolphin ever comes into its watery world it faces some significant risks. The world's smallest dolphin--it's about five feet long--the Hector's Dolphin gives birth to large babies at the very slow rate of every two to three years. And once born, it, like most other dolphin species, risks entanglement in a commercial fishing net. Dr. Liz Slooten, University of Otago, has become an authority in modeling the effects of marine mammal bycatch on their populations and is now focusing her risk analyses on the endangered Hector's Dolphin. It may be the smallest dolphin with some of the smallest numbers, but it's no small fry to her.

Giving Kurtosis a Workout

Posted Monday, June 15, 2009 by DMUU Training Team

Kurtosis is a statistical measure of a random process that is often used, but perhaps less widely understood. This blog mentions a couple of key issues and misunderstandings about kurtosis in a risk assessment model.

A high kurtosis figure is sometimes described as being associated with a distribution that has “fat tails”. However, by simply overlaying two Normal distributions with the same mean but different standard deviations (e.g. using @RISK to do so), it is visually clear that the distribution with the larger standard deviation has the “fatter tails”. However, every Normal distribution has a kurtosis of 3 (sometimes “excess kurtosis” is referred to, whereby any base calculation has three subtracted from it; this is the case when using the Excel KURT function to calculate kurtosis, for example), so the kurtosis figure does not pick up the idea that one of the distributions has more weight in the tail.

In fact, kurtosis is a simultaneous measure of the “peakedness” of a distribution and the extent to which it has “fat tails”; the Normal distribution with the larger standard deviation will have fatter tails, but will also be less peaked, and in terms of how kurtosis is calculated, these effects balance out. Kurtosis is then a bit like going for a workout, where you are required to push weights in a central direction whilst keeping your elbows up!

Another important aspect of kurtosis that is little appreciated is that the kurtosis of a binomial distribution (e.g. modelling an event risk that may or may not happen with a certain probability) increases as the probability of the event decreases. In this sense, distributions with high kurtosis figures are perhaps most easily understood as ones relating to events of low probability but high impact.

Such topics are very easy to explore with @RISK (risk analysis Monte Carlo software add-in to Excel), where the visual ability to view overlays, combined with the use of the RiskTheo functions to obtain in Excel the numerical values of statistics associated with distributions allows for a powerful environment to rapidly address such issues.

Six Sigma – Instant Gratification?

Posted Friday, June 12, 2009 by Steve Hunt

Yesterday I had an interesting conversation with a colleague who is a Six Sigma and DFSS Master Black belt who just returned from Europe where he conducted a Black Belt Training session. He pointed out how much different the Europeans are to the Americans when it comes to their expectations and commitment they show for new initiatives, particularly quality. He said when the European’s decide to introduce a program like Design for Six Sigma, they stick to it, even through the early failures and trials and tribulations. This is in contrast to many American manufacturers, who rattle through process improvement programs and philosophies like subscribing to the book of the month. (I have witnessed this firsthand, unfortunately.)

The moral of the story is do your research and make intelligent decisions on what tools you need to employ to make the necessary improvements to your company, for both its products and processes. Train your employees, implement and stick to it through good and bad. Give it the time and effort to be effective before abandoning it for another program.

Look at these programs as hiring employees - we all know hiring employees is a time consuming and costly endeavor, so by nature we don’t want to rehiring for the same position every month or year.

Rethinking Monte Carlo Simulation for Retirement Planning

Posted Friday, June 12, 2009 by DMUU Training Team

A recent article entitled “When Monte Carlo analysis meets a black swan” in Investment News addresses the criticisms Monte Carlo simulation has received for “missing the meltdown.” The author, Moshe A. Milevsky , notes that people typically seek a single number “answer” from a Monte Carlo simulation, such as the probability of meeting a single retirement planning goal. Milevsky points out that many Monte Carlo software packages do not include sensitivity or scenario analyses to drill down and determine which variables are really driving the risk inherent in the results. He proposes what amounts to a stress test – simulating what could happen under likely scenarios, and simulating again under 1-in-100 chance “black swan” disastrous scenarios. Milevsky wraps up by saying, “Instead of condemning Monte Carlo analyses for missing the meltdown, let's properly harness the full power of stochastic methods to give us tools that provide clear utility.”

I believe Milevsky makes a great point, focusing on the modeling practices rather than the tools themselves in this case. Monte Carlo simulation tools are very important for applications like retirement planning, but even the best hammer can’t help an unskilled carpenter. Tools like @RISK include sensitivity and scenario analysis, enabling easy implementation of tests under different scenarios for portfolio value, inflation, longevity, or any combination of these.

Randy Heffernan
Vice President

Using Named Ranges in Excel: Some Comments

Posted Tuesday, June 9, 2009 by DMUU Training Team

An earlier blog on Best Practice Principles in Excel Modelling generated quite some interest, as well as demand for more details on some of the points made, especially those concerning the use of named ranges risk asssessment models in Microsoft Excel. In the earlier posting, I had simply stated that (in my opinion): “Named ranges should be used highly selectively but not excessively”. Here I will expand a little more; the topic itself can be a subject of quite animated discussion within the risk analysis modelling community, with a wide set of opinions expressed. The points I make below are therefore simply my view of the topic.

In my view, named ranges are indispensible in some types of modelling situations. The most frequent of these in my experience are:

When writing VBA code (macros) that refer to ranges in the workbook (as such code almost always would do at some point), the use if names provides a much more robust way of creating flexible code, rather than referring to the range using cell references.
For general Excel modelling, it can be useful to name a small set of key ranges, so that the F5 key or the name box can be used to rapidly navigate around the model.
Where the model process is not required as a process to experiment with or modify a model, but is purely required to implement a known situation which will never be changed. However, much Excel modelling involves the process of experimenting with different approaches, and the use of named ranges in such cases can create extra complexity.

Some disadvantages of using named ranges include: that their use too early on in the risk analysis modelling process can create cumbersome structures, that it can be easy to create models with far too many names that then become poorly labelled, and the possibility to inadvertently create links between models. The management of names (such as their deletion and their scope) has traditionally been cumbersome in Excel. It is important to note that Excel’s 2007 Name Manager has radically reduced some of these disadvantages (this change being one of the most important improvements made to Excel when moving from Excel 2003 to 2007, in my opinion).

This set of points is by no means complete; a deeper discussion of modelling in Excel, including robust and readily understandable risk analysis models or option valuation and price forecasting, is contained within my book Financial Modelling in Practice.

Dr. Michael Rees
Director of Training and Consulting

There's Risk Analysis and Then There's More Risk Analysis

Posted Friday, June 5, 2009 by Holly Bailey

It must be that all the gloom in the financial sector is bringing out the gallows humor in me, because I had to laugh at this follow up on the "war games" stress testing in the British banking sector. First the Financial Times reported that risk analyses stress tests applied by regulators to British banks had accurately predicted the 2007 failure of several banks. Now the Telegraph has reported that after cracking down on lavish bonuses for bankers, the Financial Services Authority, the country's bank regulator,has paid out about $147 million in staff bonuses--which amounts to about a 40 percent hike in its payroll costs.

The Financial Services Authority defended the additional expense by saying that the heightened demands on the agency in 2008 required it to hire an additional staff 250 members--and 250 additional copies of whatever Monte Carlo software the agency uses?--and to pay for more highly skilled staff.

Remember that in 2004, with some scary reports of operations risk in some troubled banks, the regulators, apparently unable to face decision making under uncertainty on its own part, failed to crack down on the practices that eventually bought down several banks in 2007. Their answer to the accelerating crisis was more people with more statistical analysis skills. Apparently, however, more risk assessment doesn't necessarily correlate with better policy. There's risk analysis and then there's more risk analysis.

Don’t Blame the Math

Posted Thursday, June 4, 2009 by DMUU Training Team

A recent article in Bank Investment Consultant criticized the risk analysis method of Monte Carlo simulation for not taking into account extreme events like the stock market crash. According the article, a Morningstar executive states that the “bell-shaped curve that Monte Carlo simulations use” artificially assigns the probability of what happened as zero. Furthermore, the Retirement Income Industry Association calls for an update to Monte Carlo software simulators to include “a larger number of scenarios that assume greater volatility.”

These arguments demonstrate a fundamental lack of understanding of what Monte Carlo simulation is. The underlying assumption that Monte Carlo simulation itself is somehow to blame fails to recognize that Monte Carlo simulation is simply a mathematical technique that takes into account many different possible scenarios – but only within boundaries set by the user. You can’t change the underlying math behind these “what-if” calculations.

When modelers set up retirement planning or financial models, the people doing the modeling must make assumptions about the likelihood of different things happening – like the market crashing, for example. People may make those assumptions based on historical evidence or expert opinion, but it’s people who make those assumptions – not Monte Carlo simulation software. People then enter their assumptions into a Monte Carlo simulation model, setting up probability distributions to reflect their chosen likelihoods of occurrence. If the assumed volatility is insufficient, that is the fault of the modelers, not the simulation itself.

In addition, Monte Carlo simulation does not always “use” a bell-shaped curve. Uncertainty can be modeled with dozens of different probability distributions, many of them not bell-shaped. And the call to include more scenarios and volatility can easily be met by existing Monte Carlo software such as @RISK. It’s simply up the user to change the model parameters to look at more possible outcomes. The Monte Carlo simulation package won’t fight it.

It’s disappointing to see esteemed financial organizations such as Morningstar and the Retirement Income Association missing the point. Calling for changes to Monte Carlo simulation itself is not only impossible but fails to recognize the problems with modeling practices that led everyone to miss the crash.

Randy Heffernan
Vice President

Risk Analysis and War Games in Banking

Posted Tuesday, June 2, 2009 by Holly Bailey

In case you're one of the many people who believe that no one could have foreseen the banking crisis that began in the U.S. last year, the Financial Times reported just days ago that in 2004 the British banking system--specifically, the Financial Services Authority, the Bank of England, and the Treasury--did, in fact, foresee the failure of several British banks. They used risk simulation to conduct "war games" within the banking system. These war games were analogous to the recent stress tests to which U.S. banks were subjected, and they were intended to reveal the potential effects of sudden turmoil in the mortgage markets.

By revealing what banking authorities considered unacceptable risk exposure on the part of a few banks, including Northern Rock and HBOS, the 2004 risk analysis tests did fulfill their function, but bank regulators--who themselves were in did not believe they could force the banks to change their business practices. In 2007, as wholesale lending markets dried up, Northern Rock failed, and HBOS was rescued in a buyout by Lloyd's.

The moral of my story is that in banking, as in all other enterprises that require decision making under uncertainty--i.e., most of them--risk simulation and other forms of statistical analysis, dialed up properly, can provide valid forecasting--even if nobody acts on the results. Evidently the British had tuned up their Monte Carlo software better than the U.S. analysts whose simulations were tweaked to avoid the forecast of bad news.

Sources of Skewness in Risk Modelling

Posted Monday, June 1, 2009 by DMUU Training Team

The topic of skewness of an uncertain variable is perhaps one of the most fundamental in risk assessment modelling. When it is believed that a (continuous) process is symmetric, the choice of distributions to use to represent that process is generally of less consequence than when the uncertainty is asymmetric. For example, a symmetric Triangular, PERT, and Normal distribution (with appropriately selected parameters e.g. so that the means and standard deviations for each are the same) will be broadly similar; of course there are some differences, but they are generally at the margin and of little significance in many practical risk analysis modelling situations for general business purposes (though such differences can still be important in cases where extremely accurate models are required).

Here, I briefly mention some sources of skewness that arise in real-life processes, or in the associated modelling of risk:

Multiplicative processes. A process in which random variables are multiplied will create a skewed output, tending to a Lognormal distribution when many such independent variables are multiplies, and often approximated by such a distribution in any case. Such process arise in cost budgeting (e.g. the total cost estimate as the product of an uncertain volume, unit cost, and perhaps a duration), in asset price forecasting (% changes to asset values over several periods work in a multiplicative sense), and in oil reservoir modelling (uncertain reserve estimation volume estimate being the product of uncertain spatial dimensions and some additional other factors, i.e. for exploration and production).
Compound processes with event risks. When taking a pragmatic modelling approach in cost budgeting (e.g. using a Triangular distribution), one often simply assumes that the cost distribution of an item is asymmetric; that is we assume that (for unspecified reasons) the costs are more likely to be over the base estimate than below it. Often part of the underlying reason is the presence of event risks in the situation, where the occurrence of a specific event creates an additional (perhaps uncertain) set of costs in addition to a (perhaps uncertain and symmetric) base cost.
Parameter estimation for small sample sizes. When estimating a probability from a set of observations (for example 5 occurrences of an event during 100 periods, or trials), one sometimes takes the “maximum likelihood” approach (i.e. assume 5%) or otherwise assumes that there is a distribution of possible probabilities (such as a Beta distribution). Either way, for small sample sizes, the distribution of the uncertainty of the true probability is not symmetric. Examples of this were given in the earlier blog about the difficulties in estimating the probability of low probability events.

Dr. Michael Rees
Director of Training and Consulting

The Neural Network and the Click-Through

Posted Friday, May 29, 2009 by Holly Bailey

As the market for neural network software has become more and more competitive, I've been intrigued to watch the proliferation of applications for this breed of statistical analysis. In a week that has produced news of neural networks put to use to diagnose epilepsy, pick stocks, protect children from internet pornography, and predict wind power came a particularly intriguing item that might not get the attention it deserves: Penn State information sciences professor Jim Jansen and his colleague Amanda Spink of the Queensland University used a neural net for a study of internet search engines and user satisfaction. Because click-throughs mean potential sales for businesses that rely on internet advertising, the study could send search engine developers scrambling to retool their engines.

Here was the question Jansen and Spink posed: what is it about the results a search engine produces that causes the person who receives them to click through on a particular result?

Here was how they went about answering that question: they obtained data on 7 million interactions from the search aggregator Dogpile and used neural networks software to classify the purpose of the search--e.g., information gathering, navigation help--used other statistical analysis methods to relate their classifications to the number of click-throughs. Here a click-through indicates that the user was satisfied enough with the result to pursue it further, and obviously a search engine that produces more click-through responses is more commercially desirable.

I'll be interested to see the reaction to the study from the search engine community, but in the meantime I'll certainly be more aware of click-throughs in my work on commercial websites.

Fed Uses Monte Carlo Simulation for Stress Test

Posted Friday, May 29, 2009 by DMUU Training Team

The U.S. Federal Reserve recently released the results of a comprehensive assessment of the financial conditions of the nation's 19 largest banks, which hold two-thirds of American economic assets. This “stress test” was designed to determine the capital buffers required for the banks to withstand losses and maintain lending even in worsening economic conditions. Officially called the Supervisory Capital Assessment Program (SCAP), the test identified the potential losses, resources available to absorb losses, and resulting capital buffer needed.

Monte Carlo simulation was used to determine the potential losses from further defaults on loans. According to Federal Reserve Chairman Ben Bernanke, “The assessment program was a forward-looking, ‘what-if’ exercise.”

Monte Carlo simulation is one of the most widely used methods of stress testing for capital and operations risk, according to Investopedia. It takes into account variables such as interest rates, lending requirements, and unemployment. As any @RISK software user will tell you, this type of sophisticated simulation can be accomplished easily within the Microsoft Excel environment. The result of a Monte Carlo software simulation is a look at a whole range of possible outcomes, including the probabilities they will occur -- a valuable tool when stress testing.

Randy Heffernan
Vice President

Eating Time and Economic Growth: The Stability of Correlation Coefficients

Posted Tuesday, May 26, 2009 by DMUU Training Team

I have been meaning to write about the stability of correlation coefficients for a while, and have been spurred into action by a recent article in the International Herald Tribune (“Eat Quickly, for the Economy’s Sake,” May 8, 2009). The article discusses the relationship between economic growth and time spent eating, based on a recent study published by the OECD, which looked at a variety of social indicators in the 22 countries surveyed. The article finds a negative correlation between eating time and economic growth, with the U.S., Canada and Mexico being the countries in which people spend the least time eating (and with higher economic growth), and France the country where most time is spent eating (“to no one’s surprise,” according to the author).

Since I was keen to try to replicate the results of the study for myself, I downloaded the eating survey data from the OECD web-site, but then for some reason was unable to find the same economic growth figures used in the article, and so searched for similar data myself, with the result that the time period covered in measuring economic growth was different to that used in the article… the key point being that I ended up in my analysis with a positive (+16%) correlation coefficient between eating time and economic growth, rather than a negative one (…maybe a bit of relaxation does help creativity?...back to the land of dreams…).

In fact, the reality is that correlation coefficients measured between data samples are not particularly stable until the data sets become very large. The @RISK (risk analysis using Monte Carlo simulation in Excel) tool can be used to explore this issue. For example, two data sets of a given size (e.g. 22 points) containing independent distributions can be set up, and for each random sampling of the points (each iteration of a simulation), the correlation coefficient between the sample points drawn from independent distributions calculated, with this correlation coefficient being set as the output call for the simulation. Having done this, a few observations and notes can be made from such reflections and calculations:

Data sets containing only two points each will have a correlation coefficient which is either 100% or minus 100% (except in the rare case where two data points have exactly the same value), as the points are ordered either both low-high, or one is low-high and the other high-low. This already provides some intuition as to the lack of stability of the coefficient.
The correlation coefficient as measured from two independent samples of 22 data points has a standard deviation of around 20% (and a mean of zero); in roughly one-third of cases, the measured correlation coefficient between these two independent samples will be more than 20% (in either the positive of negative direction)
About 100 data points are required in each set before the standard deviation of the correlation coefficient drops to below 10%
The inverse square-root law applies: a doubling of the sample size reduces the standard deviation of the correlation coefficient to approximately 71% of the value with a smaller sample size.

Referring back to the original article, my own feeling is therefore that the relationship discussed is driven by the inherent uncertainty in dealing with small sample sizes. As the article’s author states: “Such correlations may be nothing but coincidence, of course.” As we saw when @RISK was applied to the problem, any statistical analysis can be subject to these traps.

Undaunted, the author continues: “But if the data are genuine, a contribution to world growth is rendered by any institution that enables people to eat rapidly and gain weight. Take a bow, McDonalds.”

Dr. Michael Rees
Director of Training and Consulting

The Efficient Frontier and Monte Carlo Software, II

Posted Friday, May 22, 2009 by Holly Bailey

Let's move on from yesterday's blog on the Efficient Frontier, formulated half a century ago by Harry Markowitz, to the New Frontier postulated by investment advisor Richard Machaud. Michaud is the author of Efficient Asset Management:A Practical Guide to Stock Portfolio Optimization and Asset Allocation (Oxford University Press, 2008), among other works, and now heads up New Frontier Advisors, an institutional research and investment advisory company.

Michaud's New Frontier adds further sophistication to Markowitz's ideas about optimizing investment diversification to balance risk and return by introducing resampling to the optimization process. Resampling is a method from statistical analysis that compensates for possible error by analyzing a dataset from which a subset has been portioned off and replacing values in the initial analysis with randomly sampled values from the subset.

More specifically about the New Frontier technique, Michaud adds resampling capability to Monte Carlo simulation. According to one commentator, this "allows managers to assign a greater range of probabilities to various outcomes. The goal is to produce a more realistic portfolio based on a more realistic frontier."

New Frontier now markets proprietary Monte Carlo software with a built-in resampling function to its institutional clients, and my own in-house experts tell me that resampling functionality is available in some commercial Monte Carlo Excel software as well.

The Efficient Frontier and Monte Carlo Software, I

Posted Thursday, May 21, 2009 by Holly Bailey

In my comments over the months since the economic sucker punch landed, I have been reiterating that Monte Carlo simulation is not to blame for the faulty risk assessment that brought down the derivatives markets. The assumptions that went into those risk simulation models were the source of the trouble, and that's too bad, because many versions of Monte Carlo software are flexible enough to accommodate all kinds of probability functions and timelines.

Today I came across a lucid article from IndexUniverse.com detailing just one of the ways Monte Carlo simulation can be tuned to the combined unfolding of time and risk. Tomorrow, I'll look specifically at that variation of risk analysis, but first, today, a little background.

Since Harry Markowitz won the Nobel Prize in Economics in 1990, the Efficient Frontier has been the line in the sand under which portfolio managers wiggle their toes. The efficient frontier is a major component of his Modern Portfolio Theory, which brought him the big prize. In the 1950s Markowitz was researching the idea of the present value of investments in order to optimize the return across collection or portfolio of these, and he realized that the element that was missing from ideas about present value was risk. This insight led, eventually, to his prescriptions for diversifying investments to maximize the return and minimize the risk across an entire portfolio.

Portfolio diversification is now gospel among financial planners. But gospel doesn't mean all investment advisors treat or even produce the same Monte Carlo Excel models of portfolio risk in the same way. Tomorrow, one investment advisory firm's approach to Monte Carlo and the Efficient Frontier.

Wall Street Journal Confuses Monte Carlo Simulation with Models

Posted Thursday, May 21, 2009 by DMUU Training Team

The recent Wall Street Journal article “Odds-On Imperfection: Monte Carlo Simulation” asserts that Monte Carlo simulation did not predict the market crash, and cites a chorus of critics calling for a fix to the technique. The article equates the technique of Monte Carlo simulation with the models that are using it – two very different things. For instance, the article states, “These models were supposed to help quantify and manage the risks of mortgage-backed securities, credit-default swaps and other complex instruments. But given the events of the past couple of years, it appears that the models often gave big institutions, as well as small investors, a false sense of security.”

This is true – the models for decision making under uncertainty gave a false sense of security. But that’s because the assumptions underlying the risk analysis models were flawed, not because the technique of Monte Carlo simulation was problematic. Monte Carlo simulation is simply a mathematical technique that recalculates many different possible scenarios – but only within boundaries set by the user. You can’t change the underlying math behind these “what-if” calculations.

The article comes close to making this distinction in one sentence: “Critics emphasize that the problem isn't Monte Carlo itself, but the assumptions that go into it.” It then goes on to describe efforts by firms to include “fatter tail” distributions that more accurately reflect the probability of extreme events occurring as an effort to improve Monte Carlo simulation. Tools like @RISK (risk analysis software add-in for Microsoft Excel) allow complete control over the definition of many dozens of distribution types, enabling users to create as fat a tail as they want. While these efforts make sense, it should be made clear that these are changes to underlying model assumptions, not changes to Monte Carlo simulation itself. To equate Monte Carlo simulation as a technique with the probability distributions people decide to use is to equate a carpenter’s choice of nails with his hammer.

Finally, the article cites the need to run tens or hundreds of thousands of scenarios, instead of just 100 or 1000. This too is user-defined, and tools like @RISK can run as many scenarios as desired.

Randy Heffernan
Vice President

Next