Capturing Dependencies with Correlations – Part I: Defining Correlation Coefficients

Thursday, September 2, 2010 by DMUU Training Team
When doing risk analysis simulation modeling, it’s critical to represent the fact that many variables are related. Very rarely are all variables completely independent of one another.  For example, when interest rates are low, housing starts tend to go up. This relationship is represented by a correlation coefficient, a number between -1 and 1. -1 means the two variables are perfectly negatively correlated;  that is, when interest rates go up 1 unit, housing starts go down 1 unit. A coefficient of 1 is exactly the opposite, and 0 means there is no relationship between the variables at all. Typical coefficients are between these extremes. This correlation between input variables can and must be captured in your financial risk analysis models. Even if you are using expert opinion, estimating these relationships is better than ignoring them completely.

@RISK provides an easy way to define correlation coefficients. Through the use of a slider bar, you can dynamically update coefficients between variables as well as scatter plots representing those variables. This helps you visualize and more easily define these relationships.

For a quick video demonstrating this, see:



» View short videos on recently added @RISK risk software features

Free Webcast This Thursday: The Use of the DecisionTools Suite in Biotechnology Project and Portfolio Decision Making

Monday, August 30, 2010 by DMUU Training Team
Vertex Pharmaceuticals, Inc. is a global biotechnology company based out of Cambridge, MA. The Company's strategy is to commercialize its products both independently and in collaboration with major pharmaceutical companies. Vertex's product pipeline is focused on viral diseases, cystic fibrosis, inflammation, autoimmune diseases, cancer, and pain.

Given the uncertainty of outcomes in the biotech industry, consideration of variability is an inherent part of the decision process. Often, the mean (average) is not a relevant decision criteria. This is especially true for smaller biotech companies like Vertex – the opportunity costs are extremely high because scarce capital resources would be invested elsewhere, with a higher probability of realistic return. For example, a company may reject a project which is profitable on average (positive Net Present Value) because some of the possible outcomes are unacceptable to the decision maker. Consideration of variability allows a decision maker to bring in their own risk tolerance into the decision. A similar argument applies when estimating a safety margin above a base case (e.g. in cost budgeting).

Vertex’s strategy and analytics group within the corporate finance division seeks to provide the senior management with dynamic revenue and profit forecasting methodology that helps to identify types of drugs that should be developed given a finite amount of cash and resources. A traditional financial view allows the user to identify scenarios and potential outcomes, but lacks the ability to show the range of potential values within each and every outcome. Vertex’s team uses the DecisonTools Suite to establish the average outcome, the variability of outcomes and to pressure-test risk and uncertainty of a particular scenario throughout the decision process.

Vertex’s team built a complex financial risk analysis model using @RISK to enhance its portfolio process. Monte Carlo simulation and optimization are used to analyze and optimize project and portfolio decisions, given short and long-term corporate strategy. @RISK is also frequently used throughout the business development process: simulating across multiple sales forecasts provides BD team with a range of potential outcomes, making it easy to pinpoint a particular scenario on a curve, along with its probability and value. TopRank turns the sensitivity analysis into a quick and seamless exercise, answering multiple what-if questions within minutes. Franchise and program leaders can now see a dollar effect of their program being delayed or advanced, adding supplementary indications to the development plan and even addressing the price uncertainties all at the same time. The simple interface of PrecisionTree along with tornado chart outputs makes it easy to explain the effect and importance of a particular assumption / decision to an audience with no finance background.

As the company continues to grow, adding more drugs and collaborations to its development pipeline, we will see in this free live webcast how the DecisionsTools Suite remains one of Vertex’s analytical tools of choice to enhance and guide the decision making process.

» Register now (FREE)
» View archived webcasts

Rating the Polls

Monday, August 23, 2010 by Holly Bailey
With the New York State primaries coming up September 14 and the general election on November 2, I predict that as soon as summer turns the corner into September, we'll start hearing lots and lots about polls that predict election outcomes.  To find out if there was any early discussion of polls, polling, and outcomes, I returned to my favorite election forecast site from the 2008 presidential elections, FiveThirtyEight: Politics Done Right.
 
Sure enough, there it was, a comparative rating of pollsters. This will give people like me, who tend to believe any poll just because it's covered in the news, a way to assess the poll reliability. FiveThirtyEight is the brainchild of Nate Silver, and 538 is the number of members of the Electoral College.  Silver's primary business is Baseball Prospectus, which is also fueled by Monte Carlo simulation and other risk analysis techniques, but FiveThirtyEight has done well enough for the New York Times to want incorporate it in its online coverage during the coming elections.
 
Silver's grasp of statistical analysis becomes immediately evident when you go to his page on the pollsters, and he's more than happy to discuss the statistical methods he uses to rate the pollsters--regression analysis of raw data, Monte Carlo software in an Excel spreadsheet, weighting of poll performance data, and so forth. His take on these matters may be of practical interest to any of you who use these techniques in financial risk analysis.

Elections are all about decision making under uncertainty, especially voter decisions under uncertainty, and according to Nate Silver, only polls taken within 21 days of an election are reasonably reliable.  So when the national campaigns are ramping up in October, keep one eye on the polls and one on FiveThirtyEight.  



Tackling the energy crisis by managing demand, using risk modeling software

Thursday, July 29, 2010 by DMUU Training Team
One of the side-effects of the recession appeared to be a reduction in the demand for electricity as businesses and consumers alike looked to make savings on their outgoings. However, economic recovery seems to render this trend as temporary, meaning that the global need to tackle energy-consumption is as pressing as ever.

BC Hydro, Canada's third largest electrical utility provides an interesting case study in how to  ascertain the most effective ways to tackle the gap between supply and demand of electricity in British Columbia. Trends such as an expanding population growth, and the increase in energy usage per customer, have led to a rise in the demand for electricity across the region. By legislation, BC Hydro must aim to meet these energy needs through implementing cost-effective energy conservation approaches before it can turn to increasing the supply. 

The company has set itself one of the most aggressive targets in North America, with a plan to meet almost 75 percent of its incremental load through Demand Side Management (DSM) over the next 20 years. DSM projects include compact fluorescent light promotions; subsidies for energy efficient appliances; variable speed motor promotions (for home furnaces); and promotional activity aimed at motivating customers to use less energy.

BC Hydro uses @RISK risk analysis software to measure the uncertainty around its energy conservation efforts, both at the project stage and at a higher portfolio level. Around 60 projects were analysed on a case-by-case basis, and a probability distribution around the forecast outcome was developed. @RISK helps BC Hydro to capture the level of uncertainty of the estimated savings for each individual DSM venture.

In recognition that projects do not operate in isolation, BC Hydro also uses @RISK to explore the interrelationships between key uncertainties: the participation and savings per participant, and the participation across projects. The analysis showed that if a 'conservation culture' was developed in the province, it would result in an increase in energy savings across all programmes. However, it also illustrated that, if this culture failed to materialise, the performance of all programmes will be dragged down.

Exploring uncertainty using @RISK allowed BC Hydro to find the best balance between the uncertainties of supply side resources and those of relying heavily on energy conservation. Employing Decision Making Under Uncertainty helps BC Hydro to meet both financial risk analysis and environmental risk analysis goals. As a result, it expects to meet the majority of its incremental load growth through conservation measures.

» Read the BC Hydro case study

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

@RISK Quick Tips: Correlation of Input Variables

Tuesday, July 27, 2010 by DMUU Training Team
This financial risk analysis example demonstrates the use of the Corrmat function to correlate multiple @RISK distributions. The distributions are correlated using a matrix of coefficients that specify the relationship between each pair of functions. The coefficients must be between -1 and +1, with a value of +1 indicating a perfect correlation, 0 indicating no correlation, and -1 indicating a perfect negative correlation. In this example three variables, the current US interest rate, the Pound/$ exchange rate, and the Euro/$ exchange rate, are correlated using a 3x3 matrix. Correlation matrix inconsistency occurs when a matrix that is mathematically impossible to realize is entered. If this happens, @RISK will try to adjust your matrix. However, you may have some correlation coefficients that you do not wish to be adjusted, while others are more or less free to be changed. This information may be entered using an adjustment weight matrix

» Download the example:
   CorrmatWithAdjustmentWeightMatrix.xls
» See more Financial Risk Analysis models here
 » Case Study: FiduciaryVest uses @RISK's correlation
    matrices in asset allocation modeling


Introduction, by Way of Retraction

Friday, July 9, 2010 by Holly Bailey
Just after I posted my last blog questioning a recent Investopedia column in the San Francisco Chronicle, I had a congenial note from the author of that column, David Harper.  His column compared Monte Carlo Simulation with two other methods of calculating Value-at-Risk, and I was concerned that its view of risk and risk analysis techniques was overly simplified. David   was surprised to discover that column had just appeared because he wrote it five years ago!

The five-year lag explains a lot--Monte Carlo simulation was not nearly so widely adopted or carried about by so many software tools as it is today--and I should have suspected the article was a vintage piece before I started carping.

So I happily retract my concerns to introduce to you David Harper, CPA and certified Financial Risk Manager.  In response to my comment about the attitudes and techniques that led to last year's collapse of the financial markets, David says that, now that the black swan has flown, "the crisis should implicate both HistoricalSim VaR and parametric VaR (at least multivariate normal!) and point toward Monte Carlo Sim. I've been thinking for a while that all of this [I think he means lack of accuracy in specifying risk] should really boost Monte Carlo."

Investment commentary is only one of David's activities.  He is the founder of Bionic Turtle, a business devoted to e-learning about financial risk and preparation for the certification exam for financial risk managers. This is a worthy enterprise--I was relieved to discover that there are hoops financial risk managers have to got through to be called that--and for anyone who would like to know more about quantitative techniques for risk analysis, its website is worth prowling. 

Thank you, David, for setting me straight.  

Easy, But Is It Rigorous?

Friday, July 2, 2010 by Holly Bailey
Value-at-Risk is a calculation that predicts a worst case scenario in which the maximum loss for a specific investment would be realized.  Recently the San Francisco Chronicle investment blog Investopedia, ran a short series posts on VAR.  One of the more intriguing of these demonstrated three ways of calculating Value-at-Risk for a single stock investment for more than one time period.  
 
The three methods were historical simulation, variance-covariance, and Monte Carlo simulation. What was intriguing about the comparison of methods was the observation that best choice among these methods was the variance-covariance method because it was easy. The downside of using the historical method was the need to crunch data and the downside of getting out your Monte Carlo software--no mention of using historical data to inform your model--was that the Monte Carlo method was "complex."  
 
Does that mean that risk is simple enough to require only simple statistical analysis?  And doesn't this kind of thinking encourage financial planners to take a direct but drastically reduced view of the possible outcomes of an investment?  And isn't this the same turn of mind that led to the collapse of the financial markets only a year or so ago?
 
Variance-covariance assumes volatility only in terms of standard deviation, and volatility doesn't come in one flavor or standard deviation.  Neither does risk.   

@RISK Quick Tips: Asset Price Random Walks and Options Valuation.

Tuesday, June 29, 2010 by DMUU Training Team
@RISK risk modeling software is used for a wide variety of applications in financial risk analysis forecasting, investments, and banking. This model is one example of how @RISK can help in risk analysis decision making.

Models of the prices of assets (stocks, property, commodities) very often assume a random walk over time, in which the periodic price changes are random, and in the simplest models are independent of each other. The future price level of the asset may result in some contract or payoff becoming valuable, such as in the case of financial market options. In these cases, the value of the contract (contingent payment or option) is calculated as the average discounted value of the future payoff. In the special case of European options on a traded underlying asset, the value calculated from the simulation may be compared with mathematical formulas that analytically provide the valuation, such as the Black-Scholes equation. In many more complex cases, the pertinent analytic formulas may be unknown or very complex to derive, and one may wish to rely on simulation techniques. This particular model compares the average simulated payoff for European Call and Put options with the Black-Scholes valuation.

» Download the example: AssetPrices.Options.BS.Multi.xls

@RISK Quick Tips: Discounted Cash Flow (DCF)

Tuesday, June 22, 2010 by DMUU Training Team
@RISK risk modeling software is used for a wide variety of applications in financial risk analysis forecasting, investments, and banking. Below is an application of a discounted cash flow analysis.

Discounted cash flow (DCF) calculations are a frequent example of the use of @RISK. In the example model, the sources of risk are the revenue growth rate and the variable costs as a percentage of sales. After taking into account the assumed investment, and applying a discount factor, the DCF is derived. Following the simulation, the average (mean) of the DCF is known as the net present value (NPV).

In this example, the results show that the average DCF is positive (about 40), whereas the probability of a negative DCF is about 15%. The decision as to whether to proceed or not with this project will therefore depend on the risk perspective or tolerance of the decision-maker.

This example has also been extended to calculate the distribution of bonus payments on the assumption that a bonus is paid whenever the net DCF is larger than a fixed amount (such as 50). It also uses some of the @RISK Statistics functions RiskMean, RiskTarget, and RiskTargetD to work out the average net DCF, the probability that the net DCF is negative and the probability that a bonus is paid.

» Example model: CashFlow.xls

Value-Based Management Compensation

Wednesday, June 9, 2010 by Holly Bailey
Full disclosure: I am, like so many of my friends, an investor––a small-time one--and recently, I have joined in the public outrage about bankers' bonuses and executive compensation in general. Compensation is one of the hot buttons in the debate over financial reform.  I keep wondering why compensation practices are what they are and how they could be adjusted to calm turmoil on Wall Street.

Enter Marwaan Karame, and his version of risk analysis.
 
Karame heads the New York consultancy Economic Value Advisors, which coaches major corporations on Value Based Management.  Value--long-term versus right-now profit--is the foundation of the firm's philosophy.  Its central principle is that any activity a business undertakes should increase the wealth of its shareholders--in the case of a privately held company, the number of shareholders may equal 1.
 
Karame has developed what he calls Value Based Compensation, and the goals of this are to align the self-interest of management with the self-interest of shareholders. He believes the shareholders, the company, come first.  And this means a lot of decision-making under uncertainty.  But Marwaan has a method for his management-shareholders balancing act, and it involves performance targets, statistical analysis, and risk assessment (in this case, managing probabilities of performance). His strategy involves maintaining a reserve of bonus funds and timing the payout of these rewards. 
 
The point at which Monte Carlo simulation and Monte Carlo software come in is the point at which variance between performance targets and the level and timing of reward converge. He shows his his client how to click into Monte Carlo in the Excel spreadsheet and use the software to locate the tipping point between wealth for management and wealth for shareholders. 
 
As a small--very small--shareholder, discovering that there is such a tipping point and that Karame knows how to locate it is reassuring.  Makes me feel there's someone on my side.   

Neural Nets vs. the Ripple Effect

Thursday, April 1, 2010 by Holly Bailey
About a week ago the Financial Times ran an article about a "new" investment analysis technique that could cut through turbulence in the financial markets: neural network analysis.  I thought okay, this isn't new but maybe the application is innovative.  Besides, I liked the metaphor the reporter used, a metal ball dropped in a vat of oil and the ensuing ripples that disturb the oil.
 
The article is about software developed by a Danish investment firm that turned its back on "linear" models to adopt a neural network approach that continually reclassifies investments in a portfolio and then makes suggestions about which equities to buy and which to sell. The proprietary software chews through a heap of data--prices, price-earnings ratio, and interest rates, for starters, and its performance bench mark is the Russell 1000 index. 
 
The test portfolio used to proof the method was acquired in 2007, just before the ball dropped into the oil.  For a time it seemed to hold up but then got caught in the turbulence and its undertow. It has now recovered nicely, ahead of the Russell 1000 in fact, and the asset managers are looking  for more investors. This is a sweet success story, especially given the demon turbulence looming over the project and the fact that the assets are apparently owned by the Danish state pension plan.

I understood the use of neural network software to counter nonlinear events like market turbulence, and I understood the continual classification and reclassification.  But I was intrigued that nowhere in the article was there a mention of risk, risk analysis, or even risk assessment.  Maybe it was there all the time, incorporated in the proprietary software, and maybe it just wasn't mentioned.  Certainly the asset managers who developed the program were aware they were at risk--they were chewing their nails as their fund slid down right beside all the other funds that were dropping in value.  But assessing risk doesn't seem to have been a factor in the firm's new defense against mayhem in the markets.  
 
So.  Is it time to shut down your Monte Carlo software?  I don't think so. . . .   

Risk in the financial sector – have we learned any lessons?

Monday, March 22, 2010 by DMUU Training Team
As part of his pre-Budget report in December, the UK chancellor, Alistair Darling, announced a one-off super-tax on bankers' bonuses. This followed ongoing threats by bankers that they will leave the UK if their (bonus) earning potential is curtailed.

Unsurprisingly, this angered the British tax-payer who, thanks to the excessive risks taken by the banks, is now the proud owner of several formerly publicly-owned national banks.  As a result of this and the severe recession that followed, for a while it seemed as though the financial sector would have to change.  However, the current headlines suggest that, whilst the financial crisis was certainly a sharp shock, it may have been too short to ensure that measures were put in place to ensure it never happened again.

The key factor to understand is the grasp that money has over financial institutions.  In fairness this is as it should be – after all their raison d'etre is to make money.  However, this has developed into a culture of 'profit-at-any-cost' that is inherent throughout almost all financial organisations.  One outcome is inappropriate incentive structures that reward short-term income-generation over and above any other activity.  Another repercussion, particularly over the past few boom years, has been an increased tolerance of risk.

Over the past two years or so, many risk departments will have flagged up levels of uncertainty that, in previous times, would have been unacceptable.  For various reasons, much of this advice has been ignored.  Frustrating at the time, in light of events of the past few months, this must now seem inexcusable to risk managers, both within and outside the financial sector.  Many of these people will know that sophisticated risk analysis tools are available to enable them to 'measure' the likelihood of an event occurring and the severity of its effects.

The accuracy of the results depends on the quality of the data input.  It also hinges on the ability of the financial sector to adopt a realistic attitude to risk.  And, to quote City minister Lord Myners, this means that bankers must 'live in the real world'.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Rumors of Death

Monday, March 15, 2010 by Holly Bailey
Allan Roth, who writes a blog for CBS Money Watch called "The Irrational Investor," recently asked his readers a rhetorical question: Is Financial Monte Carlo Simulation Dead? Since rhetorical questions demand an answer in less time than it takes the questioner to draw breath, Roth obliged. 
 
While expressing sympathy for the investors who were victims of poor risk assessment and forecasting when the financial markets shook themselves down to rubble in 2008, Roth is taking a very politely defensive swing at one of the many critics of risk analysis who have turned up the volume since then--one Jim Otar of Otar Retirement Solutions and the author of Unveiling the Retirement Myth.  

Roth is an experienced user of Monte Carlo software who knows the pitfalls of overoptimistic assumptions.  He says he finds 99 percent of the Monte Carlo models he's see over the years to be inadequate because of this flaw.  Jim Otar, for his part, finds other flaws as well: in the generation of randomness and trends and in the sequence of returns. Otar's modeling method does not rely on randomness but on a century's worth of historical data. 
 
Our two worthy opponents put their models up against one another in a match that crunched identical inputs.  Their models produced very, very similar results, apparently satisfying each analyst as to the superiority of his method.  But while Roth said nice things about Otar and his model, he pointed out the limitations of relying on historical information alone. In other words, he doesn't concede.
 
For any kind of retirement planning models, he says, the cure to flaws is conservative input. Then he giddily sends his readers to one of those rudimentary online Monte Carlo calculators that investment firms love to offer their clients. 
 
Rumors of this death are greatly exaggerated.  

Quantitative risk assessment under utilised for infrastructure projects

Friday, March 12, 2010 by DMUU Training Team
Why is it that most of the high profile projects managed by the government in the UK all ultimately become beset by problems? A number of projects jump to mind – the Millennium Dome, Wembley Stadium and currently the NHS IT. All three have been plagued by developmental delays and financial mismanagement.

Recently, yet another worthy, but ambitious project has been announced – the North-South high speed rail line to connect London to Scotland. One wonders if the government undertakes detailed quantitative project risk analysis for its infrastructure initiatives?

A good example to highlight in this context is ENGCOMP, a Saskatchewan-based engineering consulting firm that has worked with the Canadian Department of National Defence (DND) to help define budgets for the fourth phase of construction of its Fleet Maintenance Facility at Canadian Forces Base Esquimalt in Victoria, British Columbia. Using @RISK, a Monte Carlo simulation tool, ENGCOMP helped the DND define and secure budget approval from the Federal Government’s Treasury Board. The consultancy firm was able to estimate the impact of the variability and uncertainties pertaining to risks, costs and scheduling. This assessment enabled it to estimate the project risk budget or the risk reserve and schedule contingency, which were both factored in when defining the total project cost of the infrastructure project.

The fact is, in the world of business, risk is inherent and unavoidable. Whilst one cannot completely control risk, one can certainly help reduce uncertainty, greatly increasing the chances of project success. For instance, a key finding of the project risk analysis conducted by ENGCOMP was that, taking into account all the risk and uncertainties on the project, there is an 85 per cent chance that the Fleet Maintenance Facility project will be completed in January 2014. A fairly positive result for the DND, given the scale and complexity of this project in question.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Palisade is proud to announce our first Health Risk Analysis Forum in San Diego on March 31st 2010

Wednesday, March 10, 2010 by DMUU Training Team



Why attend?

This one-day forum is a great way to find out how others in the Healthcare Industry are using our software, as well as to learn new approaches to the problems Healthcare professionals face every day. We will have six software training sessions, and six real-world case studies presented by industry experts covering risk and decision analysis from all angles specific to the Healthcare sector.

You will also see how new versions of @RISK, PrecisionTree, RISKOptimizer, TopRank, NeuralTools, StatTools, and other Palisade software tools work together to give you the most complete picture possible in your situation.

Who should attend?


Professionals in risk and financial analysis in: Care Equipment & Services, Pharmaceuticals, Biotechnology & Life Sciences, Hospital Care & Management, or related services

How much?


For a limited time, the cost for attending the Health Risk Analysis Forum is has been discounted $100.

$295 covers all sessions, continental breakfast, lunch and a cocktail networking reception. Attendees will also receive a welcome package that includes a 15% discount on their next software purchase.

Please contact Jameson Romeo-Hall at jromeo-hall@palisade.com if you are interested in attending.

Location
The Westin Gaslamp Quarter
910 Broadway Circle
San Diego, CA 92101
(619) 239-2200

Book your room at a discounted rate (subject to availability.)


Pensions – The Ticking Time Bomb

Monday, March 1, 2010 by DMUU Training Team
Both the Conservative Party and the Labour Government have indicated that they will raise the state pensions age of men and women to help reduce the UK’s national debt.  In addition, more and more employers in the private sector are closing good pension schemes. The Association of Consulting Actuaries’ (ACA) recent survey on pension trends has revealed that 59% of employers are set to review pensions ahead of 2012 and 24% of employers will consider pension benefit reductions when they have to auto-enroll all employees into a scheme.

With taxes on business and individuals likely to rise over the next few years, it is difficult to see anything other than a deteriorating climate for pension savings unless there is a radical change of approach, says the ACA. It has proposed a standing Pension Commission that will challenge the legal and regulatory hurdles standing in the way of sensible long-term pension designs.

Perhaps, a more in-depth risk analysis may help the ACA make a stronger case to the government. As a related example, in the US, the Society of Actuaries and the Casualty Actuary Society, sponsored a research project with the Illinois State University to develop a model for projecting economic indices such as interest rates, equity price levels, inflation rates, unemployment rates, and real estate price levels. The model was created using Palisade’s @RISK and Microsoft Excel. In fact, @RISK’s built-in probability distribution functions, correlation matrices, and simulation results were essential to the study.

The UK ‘pensions’ landscape is set to undergo tremendous change, which will impact each and every one of us. Using scientific, risk analysis techniques, actuarial industry bodies can develop a strong argument and lobby the government so that informed policy decisions are made that are right for both the financial health of the nation and its citizens.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

What Should You Get From a Simulation? Part 1

Thursday, February 25, 2010 by DMUU Training Team
I read an interesting article on the causes of the Global Financial Crisis by John B. Taylor. Although the topic is interesting enough already, especially for a member of a risk analysis-specialising company, something else caught my eye. I have observed in training workshops, onsite consulting and now academic papers a phenomenon regarding probabilistic modelling. Many of those using the methods don’t understand what they should actually be getting from the methodology. There is an intellectual leap from the deterministic to the probabilistic that sometimes does not get made. This limits the usefulness of Monte Carlo simulation, and the value of performing such statistical analyses.

Back to the article which spurred me to write this blog in the first place. Or rather, the graph. Yes a single graph of housing starts vs. time (and its brief description) leapt out at me. One of the lines on the graph was claimed to show model simulations of housing starts using the actual interest rate, compared to the interest rate ‘predicted’ by the Taylor Rule and a third line showing actual data.

So what’s the problem?

The problem is that simulation techniques should not be used to create a single value. The single ‘simulation’ line implies a single modelled/returned value for each time period. This is deterministic modelling. There may be a particular scenario that has been modelled, but it certainly isn’t a simulation that is being represented by that single line. Simulations produce thousands of data, observed values and their associated percentiles as well central moments (mean, variance etc.). Not just one value (sorry Value at Risk – that includes you too) that can be plotted as a single line. I would guess that if a simulation were run as I understand the term then the line in the chart was probably constructed using the simulated means. But I shouldn’t be guessing.

This is far from the only time I’ve seen simulation results reduced to a single entity. I have heard from clients in the past “the simulation gave $X” with little to no context around it, and this is supposed to both mean something to me and to their customers and help to make better decisions under uncertainty…

In the next blog I will explore this idea further and discuss the sorts of results that should be gleaned from a simulation. In particular, why narrowing simulation results down to a single number is counterproductive to healthy business practices.


Rishi Prabhakar
Trainer/Consultant

Opportunity Costs of Risk Analysis

Friday, February 12, 2010 by Holly Bailey
Merck's Art Misyan, currently Director, Financial Evaluation and Analysis at the company and a longtime practitioner of risk analysis and decision evaluation, has offered some cogent comment in response to my blog about the calculating the opportunity costs of risk analysis in making decisions under uncertainty:
 
"In the Vail Daily News comment, they refer to the cost of being the second entrant.  The impact of losing your innovative advantage can be somewhat quantified in a sales forecast, for example: if our launch is delayed, or if we are no longer the first entrant, then there is an EPS impact of $X.
 
For day-to-day risk management activities, quantifying opportunity costs is more challenging.  Sometimes the best decision is the one you didn't make, and other times it costs you either in ongoing transaction costs, deal premiums, etc.  For example, transaction costs can rise if the markets become more illiquid over the course of a trading day (say you're trying to trade Far East currency, but now it's late in the day Eastern Standard Time).  Or, if you are executing a large-sized deal but don't place the order until late in the day - and the trade has to happen.  So, hypothetically, you could calculate the impact of transaction costs, based upon average deal size and bid/offer spread at a time of day.
 
As a finance representative on the deal team, you are trying to help management arrive at quick decisions with the best available information, while understanding the potential risks. You don't want to be the "speed bump" in the process (again, very difficult to quantify).  As part of the economic analyses, we summarize as many risks as possible, as well as a list of potential events that could impact our assessment.  After management has reached a decision, we will revisit the numbers if or when these events occur over the course of the due diligence process."

Words from the wise to the wise.

Data Issues Part 2

Tuesday, January 19, 2010 by DMUU Training Team
In my last blog I mentioned a ‘fact’ about data that came up during a recent public training course (Decision-Making and Quantitative Risk Analysis). This fact stuns me every time I think about it, and certainly floored me the first time I encountered it. So many companies just don’t have it.

Data, that is. Historical data from completed projects, sometimes billion-dollar projects, is simply not collected especially in resources and infrastructure cost estimation. Instead every risk is re-estimated from scratch in every new project based entirely upon an estimator’s recollections or guesses. This is not a suggestion that estimators don’t know what they’re talking about, rather that the benefits of adding historical data to the analysis far outweigh the cost of gathering the information in the first place.

I first worked in the banking sector, hence my surprise to learn of this lack of data storage in certain areas of risk analysis. Project cost estimation, especially in resources and infrastructure – I’m talking to you. In financial circles there are literally millions of data points collected daily across the entire organisation. Gathering data (and then analysing it for some benefit) is simply ‘what we do’, and this process isn’t challenged. Some of the data is quite ‘small’, such as the number of seconds a particular caller was kept on hold before being answered, and others are quite ‘big’, such as multi-million dollar losses due to fraudulent activities. Regardless, it’s all kept in the knowledge that information is power – in this case the power to make intelligent decisions in the future.

How can you judge the efficacy of an estimation process (workshops etc.) if you don’t track the final observed outcomes specifically to make such a judgment? Well, you can’t. And that leaves your company’s risk and decision assessment process in limbo. Without measurement there can be no process improvement or corporate learning. Are you ‘passing’ or ‘failing’ with your use of Monte Carlo simulation via risk analysis software?

Generally the observed outcomes for risks in models will be near the estimated value, and this is to be expected. However the main role of risk analysis is to adjudge exposure to the unexpected. Far too many cost estimation models have very little volatility in their line items. I am very curious to know just how often the realised value of a given line item is outside the range of “possible values” as defined in the model. And what about the total project costs overall? This hints at and leads to the big question which is what could/should be done with such data if it were to be recorded?

I shall address these questions in the next blog. I know you’re excited to find out!

Rishi Prabhakar
Trainer/Consultant

The CDO Is Back in the Spotlight

Thursday, December 24, 2009 by Holly Bailey
About this time last year the term "CDO" began to make regular appearances in the news.  
The so-called "Collateralized Debt Obligations" were commonly blamed for sending an already shaky finance sector into exponential decline.  
 
Today "CDO" returned to the front page of the New York Times in article reporting an investigation by Congress, the Securities and Exchange Commission, and the Financial Industry Regulatory Authority into the question of whether Goldman Sachs and other investment banks that sold the CDOs engaged in dirty dealing against the clients who bought the synthetic debt packages.  The concern of the investigators is that Goldman, Deutsche Bank, Morgan Stanley and others knew that the CDO investments would sour and profited from short selling the stock of companies that bought the investments.
 
The investigation is still in its early stages, and those involved in it are playing zipper lips. Whether or not the investment banks broke any securities laws is still to be discovered. But in the meantime, I find the complexities of this kind of trading daunting and am fascinated to think about the minds that created the deals.  How did the financiers decide what to charge for the CDOs, how to determine their value-at-risk, and, if they did sell short against their customers, when to make the trades?  Obviously, in addition to some very finely tuned risk analysis and a great big Monte Carlo software package, a love of brinksmanship was necessary.  
 
This is the stuff of paper chase novels.  One former Goldman Sachs dealer has capitalized its on its sales potential with How I Caused the Credit Crunch--how much risk assessment was involved in that move!--and as it unfolds, the current Times story promises just as much page-turning fun.