Making Optimal Choices, or Just Making Choices? Part 2

Thursday, March 18, 2010 by DMUU Training Team
In my last blog entry I introduced the notion that optimal decision making wasn’t ‘on the radar’ for many clients in Australasia, and laid out a couple of ideas why. I too once focussed on Monte Carlo simulation rather than decision evaluation, but last year the most obscure event changed that.

Call me a nerd of you will, but I like modelling problems in Excel. There is skill involved in setting up a problem such that the model assumptions aren’t too gross, and an art to making the model elegant. This elegance can be very important to optimisation problems, but more on that later. My first homemade optimisation problem was generated by motorcycle racing! MotoGP, to be precise. A friendly tipping competition with friends was formed at the start of the 2009 season with the following structure:
  • Entrants played the role of Team Manager.
  • Team Managers had a fixed budget to spend on riders.
  • Either a few good riders could be purchased, or many lesser riders, or something in between.
  • The team that had accumulated the most points at the end of the season was the winner and received kudos!

Although the future results could not be known of course so I set up and ran the optimisation with Evolver after the event to see what the optimal team selection would have been. Historical data could have been used to discover the type of rider mix that tended to be optimal and thus make an informed decision for this competition. The risk in having only a few riders was that any misfortune would have a big negative impact on the points won, whereas a team consisting of many (cheaper) riders was less likely to suffer such a fate. This downside scenario will be modelled into the 2010 MotoGP Team Manager predictive, optimised model (currently in production)!

What has this to do with the corporate world? Replace “team” with portfolio and “riders” with “assets”, “shares” or “projects” and you have a classic portfolio optimisation model. I hadn’t created this model with business applications in mind but I realised that was precisely what I was doing. An instant later I realised just how useful Evolver would be in many decision scenarios even though it doesn’t incorporate uncertainty (RISKOptimizer does).

In the next instalment I will further explore some practical applications for Evolver and you’ll see just how universally appropriate it can be.

» Making Optimal Choices, Part 1

Rishi Prabhakar
Trainer/Consultant

New Approaches to Risk and Decision Analysis

Wednesday, March 17, 2010 by DMUU Training Team


Risk analysis and decision-making tools are relevant to most organisations, in most industries around the world.  This is demonstrated by the speaker line-up at this year's European User Conference, an event at which we believe it is important to bring together customers from a wide range of market sectors.

We are holding 'New Approaches to Risk and Decision Analysis' at the Institute of Directors in central London on 14th and 15th April 2010.  As with previous years, the programme aims to provide everyone attending with practical advice to enhance the decision-making capabilities of their organisation.  Customer presentations, which offer insight into a wide variety of  business applications of risk and decision analysis, include:
  • CapGemini: Faldo's folly or Monty's Carlo – The Ryder Cup and Monte Carlo simulation
  • DTU Transport: New approaches to transport project assessment; reference scenario forecasting and quantitative risk analysis
  • Georg-August University Research: Benefits from weather derivatives in agriculture: a portfolio optimisation using RISKOptimizer
  • Graz University of Technology: Calculation of construction costs for building projects – application of the Monte Carlo method
  • Halcrow: Risk-based water distribution rehabilitation planning – impact modelling and estimation
  • Pricewaterhouse Coopers: PricewaterhouseCoopers and Palisade: an overview
  • Noven: Use of Monte Carlo simulations for risk management in pharmaceuticals
  • SLR Consulting: Risk sharing in waste management projects - @RISK and sensitivity analysis
  • Statoil: Put more science into cost risk analysis
  • Unilever: Succeeding in DecisionTools Suite 5 rollout – Unilever's story
We will also look at the recently-launched language versions of @RISK and DecisionTools Suite, which are now available in French, German, Spanish, Portuguese and Japanese.  Software training sessions will provide delegates with practical knowledge to ensure they can optimise their use of the tools and implement business best practise and methodologies.

With over 100 delegates from around the world attending, the event is also a good opportunity to network and knowledge-share with risk professionals from around the world.

» Complete programme schedule, more information on each presentation,
   and registration details



Quantitative risk assessment under utilised for infrastructure projects

Friday, March 12, 2010 by DMUU Training Team
Why is it that most of the high profile projects managed by the government in the UK all ultimately become beset by problems? A number of projects jump to mind – the Millennium Dome, Wembley Stadium and currently the NHS IT. All three have been plagued by developmental delays and financial mismanagement.

Recently, yet another worthy, but ambitious project has been announced – the North-South high speed rail line to connect London to Scotland. One wonders if the government undertakes detailed quantitative project risk analysis for its infrastructure initiatives?

A good example to highlight in this context is ENGCOMP, a Saskatchewan-based engineering consulting firm that has worked with the Canadian Department of National Defence (DND) to help define budgets for the fourth phase of construction of its Fleet Maintenance Facility at Canadian Forces Base Esquimalt in Victoria, British Columbia. Using @RISK, a Monte Carlo simulation tool, ENGCOMP helped the DND define and secure budget approval from the Federal Government’s Treasury Board. The consultancy firm was able to estimate the impact of the variability and uncertainties pertaining to risks, costs and scheduling. This assessment enabled it to estimate the project risk budget or the risk reserve and schedule contingency, which were both factored in when defining the total project cost of the infrastructure project.

The fact is, in the world of business, risk is inherent and unavoidable. Whilst one cannot completely control risk, one can certainly help reduce uncertainty, greatly increasing the chances of project success. For instance, a key finding of the project risk analysis conducted by ENGCOMP was that, taking into account all the risk and uncertainties on the project, there is an 85 per cent chance that the Fleet Maintenance Facility project will be completed in January 2014. A fairly positive result for the DND, given the scale and complexity of this project in question.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Palisade is proud to announce our first Health Risk Analysis Forum in San Diego on March 31st 2010

Wednesday, March 10, 2010 by DMUU Training Team



Why attend?

This one-day forum is a great way to find out how others in the Healthcare Industry are using our software, as well as to learn new approaches to the problems Healthcare professionals face every day. We will have six software training sessions, and six real-world case studies presented by industry experts covering risk and decision analysis from all angles specific to the Healthcare sector.

You will also see how new versions of @RISK, PrecisionTree, RISKOptimizer, TopRank, NeuralTools, StatTools, and other Palisade software tools work together to give you the most complete picture possible in your situation.

Who should attend?


Professionals in risk and financial analysis in: Care Equipment & Services, Pharmaceuticals, Biotechnology & Life Sciences, Hospital Care & Management, or related services

How much?


For a limited time, the cost for attending the Health Risk Analysis Forum is has been discounted $100.

$295 covers all sessions, continental breakfast, lunch and a cocktail networking reception. Attendees will also receive a welcome package that includes a 15% discount on their next software purchase.

Please contact Jameson Romeo-Hall at jromeo-hall@palisade.com if you are interested in attending.

Location
The Westin Gaslamp Quarter
910 Broadway Circle
San Diego, CA 92101
(619) 239-2200

Book your room at a discounted rate (subject to availability.)


What Should You Get From a Simulation? Part 3

Tuesday, March 9, 2010 by DMUU Training Team
In the last two blogs I have challenged the idea that simulation results can be boiled down to a single statistic with any positive benefit. The context of a statistic is incredibly important, which is another reason why many statistics and charts/tables should be reported on, not simply one figure. And here’s a compelling reason why.

Consider two competing, similarly-sized projects, of which a company can only pursue one. Now let’s say this company would like to take on the project that has the “least risk”. If they are only familiar with generating the P90 for the total project cost they will be forced to select the project with the lowest P90. But what if the key drivers for exceeding the P90 are easier to mitigate in one project compared to the other? Perhaps the project with the lower P90 also has a higher P95 or P99 – this means the catastrophic failure is actually greater despite a lower P90 and is the mathematical equivalent of “when things go bad, they go really bad”. Not all P90s are created equally! Such an adverse outcome might sink a smaller company where a larger one could wear the loss. The context of the company running the analysis also impacts the context of the analysis itself.

So you can see not only do simulations generate results with which informed decisions can only be made if approached holistically, but if the language used is restrictive this outcome will never be achieved. Risk analyses are a necessary part of business because most of us wish to minimise the chance that something bad will happen, quite simply. Even if a manager tells you they “want the P90” what they are really asking is “tell me about the risk we’re facing”. The answer to this fundamental question is not found in a single figure taken from a simulation, but in a range of charts and tables which require correct interpretation.

More so, Monte Carlo simulation itself is only one piece of the risk and decision assessment pie. Decision modelling and optimisation, predictive modelling and statistical analyses should also form part of the quantitative approach to uncertainty. There is life beyond just risk simulation software, and I intend on exploring that in future blogs.

» Part 1
» Part 2

Rishi Prabhakar
Trainer/Consultant

Use of @RISK for Probabilistic Decision Analysis of a Manufacturing Forecast in an Environment of High Uncertainty

Monday, March 8, 2010 by DMUU Training Team
This Thursday, 11 March 2010 at 11am ET, Dr. Jose A. Briones, SpyroTek Performance Solutions, will present a free live webcast entitled, Use of @RISK for Probabilistic Decision Analysis of a Manufacturing Forecast in an Environment of High Uncertainty

Profitability projections in a manufacturing environment are directly tied to how the sales forecast fits with the capability of the operation. When a company has a large portfolio of products with very different operational production rates, the manufacturing capacity of the plant will be significantly impacted by the product mix to be produced. This in turn will have a radical effect on the output of the plant and the allocation of the fixed cost of production. In this case we present an example where a company is trying to decide how best to balance the sales of certain families of products to maximize revenue, maintain a diverse product line, and properly price each individual product based on the impact to the manufacturing schedule and fixed cost allocation.

» Register now for this FREE live webcast
» View archived webcasts

How can the UK public services prepare for unpredictable, extreme weather?

Friday, March 5, 2010 by DMUU Training Team
The UK Met Office is not going to ‘live down’ its weather forecast of a ‘barbeque Summer and a mild Winter’ for 2009, anytime soon. There was ample rain through the Summer, the Cumbrian region saw severe flooding in November and now the nation is gripped by sub-zero temperatures not experienced for more than 30 years.

The inaccurate weather forecast is not a criticism of the Met Office. Forces of nature cannot be controlled, but these severe weather conditions do highlight the need for a more risk-led approach to public service planning. As we are seeing, the lack of planning to combat the current Arctic conditions engulfing the nation has thrown the country in turmoil, not to mention the substantial losses incurred by businesses. 

Global Warming is now often touted as the reason for such vagaries in weather, which according to environmentalists is set to intensify in the coming years. There is a very strong case for the government to undertake a scientific, risk-led approach to assess the potential effects of extreme weather, so that the required planning and realistic fund allocation can be made to deal with unforeseen weather situations. 

For instance, Halcrow Group Ltd, specialising in providing planning, design and management services for infrastructure development, works very closely with the UK Environment Agency on its Flood Defense programme. It conducts risk analysis on several of the Agency’s projects, using Palisade’s @RISK. Through flood risk management, the UK’s Environment Agency can reduce the probability of flooding from rivers and the sea through the management of land, river systems, and flood and coastal defenses. This also works to helps to reduce the damage floods can do through effective land use planning, flood warning and emergency responses.

There is now a dire need to extend this risk analysis-based approach beyond just flood defense, so that pre-emptive actions can be taken to reduce the adverse impact of extreme weather on the nation.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

What Should You Get From a Simulation? Part 2

Wednesday, March 3, 2010 by DMUU Training Team
Where I left off last time was lamenting the use of Monte Carlo simulation to create a single value (statistic etc.) from a model. It might still not be clear why this is anathema to me, so here goes.

A simulation is not a number. It’s not one possible (future) outcome – that’s a scenario. Monte Carlo simulation is a methodology for understanding one’s exposure to outcomes not situated close to the central tendency of the process/project in question. Note the plural “outcomes”. Risk analysis, when done properly, should let you know essentially all possible outcomes and how likely they are for your model. Output from a simulation can include a plot of means (over time), or P5s, or P95s, or the mean ± one standard deviation or any number of statistics. But that’s not plotting a simulation! Let’s not give a minimalist graph too much credit.

Such statements also perpetuate the idea that simulation is only used for creating means (or other centrally tending statistics) and ignores the wealth of information available. Risk simulation software exists to help you do risk analysis which must include not only several statistics but also sensitivity information. It is all too easy to turn a risk assessment into a hunt for a regularly asked for percentile (such as the P90) and there ends the task. I see this a lot, especially in project cost estimation where the pressure both from management and regulatory bodies is to accurately estimate some large percentile. Once found there is usually scant further risk analysis.

Nothing good ensues. When risk analyses are run “to get ‘the’ number” they become simply another box to tick in a process and ultimately any benefits (perceived or actual) will be forgotten and lost to the ages. The notion of context is also lost. No single number by itself really means anything, or at least shouldn’t mean anything to a decision maker. I have often heard phrases like “the model returned/gave $1.2m” followed by an audience nodding in agreement. Huh? Which statistic are you talking about there, and how about reporting a few other numbers around it to place that $1.2m somewhere meaningful?

In the next installment I will look further into this issue of context and hopefully prove the necessity of an holistic approach to understanding and reporting simulation results.

» Part 1

Rishi Prabhakar
Trainer/Consultant

Pensions – The Ticking Time Bomb

Monday, March 1, 2010 by DMUU Training Team
Both the Conservative Party and the Labour Government have indicated that they will raise the state pensions age of men and women to help reduce the UK’s national debt.  In addition, more and more employers in the private sector are closing good pension schemes. The Association of Consulting Actuaries’ (ACA) recent survey on pension trends has revealed that 59% of employers are set to review pensions ahead of 2012 and 24% of employers will consider pension benefit reductions when they have to auto-enroll all employees into a scheme.

With taxes on business and individuals likely to rise over the next few years, it is difficult to see anything other than a deteriorating climate for pension savings unless there is a radical change of approach, says the ACA. It has proposed a standing Pension Commission that will challenge the legal and regulatory hurdles standing in the way of sensible long-term pension designs.

Perhaps, a more in-depth risk analysis may help the ACA make a stronger case to the government. As a related example, in the US, the Society of Actuaries and the Casualty Actuary Society, sponsored a research project with the Illinois State University to develop a model for projecting economic indices such as interest rates, equity price levels, inflation rates, unemployment rates, and real estate price levels. The model was created using Palisade’s @RISK and Microsoft Excel. In fact, @RISK’s built-in probability distribution functions, correlation matrices, and simulation results were essential to the study.

The UK ‘pensions’ landscape is set to undergo tremendous change, which will impact each and every one of us. Using scientific, risk analysis techniques, actuarial industry bodies can develop a strong argument and lobby the government so that informed policy decisions are made that are right for both the financial health of the nation and its citizens.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

New business planning – measuring feasibility

Tuesday, February 23, 2010 by DMUU Training Team
The latest Business in Britain survey from Lloyds TSB Commercial shows that the UK's commercial enterprises are regaining confidence.  The six monthly report charts the performance of 1,732 UK companies and their views on prospects for the coming year. Its most recent business confidence shows that expectations for both sales and orders have started to recover. The balance of firms anticipating an upturn in sales has climbed to 21% - from just 1% six months ago.   And hopes for orders are also looking brighter. The balance expecting order levels to rise over the coming six months has climbed to 23%, from just 6% in the last survey.

But companies planning major new business drives for 2010 would do well to follow the example of Thales UK, which uses @RISK  to enable it to assess commercial feasibility of potential new business wins. @RISK's in-depth risk analysis ensures the leading provider of mission-critical electronic information systems for aerospace, defence and security markets around the world, is fully informed when making business-critical decisions.

Thales operates in a highly competitive environment, with technologically advanced countries presenting tough opposition when it tenders for contracts. It must continually develop highly sophisticated equipment that is robust and failsafe to meet the stringent demands of its customers. Bringing products of this calibre to market is costly in terms of time and resource, so for every competitive new business opportunity, Thales must be confident that it has a reasonable chance of success.

Using Monte Carlo analysis to show all potential scenarios and the likelihood that each will occur, @RISK enables Thales to calculate the competitiveness of complex markets, measure probabilities for project costs, quantify rate of return, and even account for the effects of cumulative business, thereby providing decision-makers with the most complete picture possible.  From this risk analysis, Thales can make an informed decision on the commercial viability of the potential new business offered.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Free Webcast This Thursday: “Cost-Effectiveness Analysis of Patient Care using The DecisionTools Suite”

Tuesday, February 16, 2010 by DMUU Training Team
On Thursday, February 18, 2010, Prakash Shrivastava will present a free live webcast entitled. "Cost-Effectiveness Analysis of Patient Care using The DecisionTools Suite"

Cost-effectiveness analysis is often used to evaluate effectiveness of medical interventions and is one of the main topics in healthcare research. This analysis requires data evaluation, including building and testing decision analysis models. This free live webcast will illustrate how such models are easily built using the DecisionTools Suite. The first part of the webcast will introduce process, cost drivers and measures in healthcare. In the second part, cost-analysis examples will be presented. The webcast will conclude with a sensitivity analysis example.

Prakash Shrivastava is a Principal at Strategic Management International, Inc. He specializes in Research, Analysis and Simulation of Control Systems, Business Models and Processes. He worked in Automotive Industry for over 22 years and Aerospace Industry for 7 years. He holds a Masters Degree from Indian Institute of Technology, Bombay, a Doctoral degree from Princeton University, NJ and a Masters degree in Management of Technology from RPI.

» Register now (FREE)
» View archived webcasts

Opportunity Costs of Risk Analysis

Friday, February 12, 2010 by Holly Bailey
Merck's Art Misyan, currently Director, Financial Evaluation and Analysis at the company and a longtime practitioner of risk analysis and decision evaluation, has offered some cogent comment in response to my blog about the calculating the opportunity costs of risk analysis in making decisions under uncertainty:
 
"In the Vail Daily News comment, they refer to the cost of being the second entrant.  The impact of losing your innovative advantage can be somewhat quantified in a sales forecast, for example: if our launch is delayed, or if we are no longer the first entrant, then there is an EPS impact of $X.
 
For day-to-day risk management activities, quantifying opportunity costs is more challenging.  Sometimes the best decision is the one you didn't make, and other times it costs you either in ongoing transaction costs, deal premiums, etc.  For example, transaction costs can rise if the markets become more illiquid over the course of a trading day (say you're trying to trade Far East currency, but now it's late in the day Eastern Standard Time).  Or, if you are executing a large-sized deal but don't place the order until late in the day - and the trade has to happen.  So, hypothetically, you could calculate the impact of transaction costs, based upon average deal size and bid/offer spread at a time of day.
 
As a finance representative on the deal team, you are trying to help management arrive at quick decisions with the best available information, while understanding the potential risks. You don't want to be the "speed bump" in the process (again, very difficult to quantify).  As part of the economic analyses, we summarize as many risks as possible, as well as a list of potential events that could impact our assessment.  After management has reached a decision, we will revisit the numbers if or when these events occur over the course of the due diligence process."

Words from the wise to the wise.

Free Webcast this Thursday: "Simulating the U.S. Economy: Where will we be in 100 years?"

Tuesday, January 26, 2010 by DMUU Training Team
There is an assumption that drives all of our expectations for how our economy will be in the future.  That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

In this free webcast, Dr. William Strauss models the next 100 years, based on the last century's data. The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? The experiment uses @RISK’s risk analysis and Monte Carlo techniques to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization software) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

Palisade is pleased to host this presentation from Dr. William Strauss.

William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics). Read more of Dr. Strauss' bio here.

» Complete abstract of "Simulating the U.S. Economy: Where will we be in 100 years?" 
» Register now (FREE)  
» View archived webcasts

Data Issues Part 3

Tuesday, January 26, 2010 by DMUU Training Team
In Part 2 of this series I finished by asking what should be done with historical data, now that we have decided that storing it is probably a good idea. I won’t keep you waiting any longer.

Auditing and calibration of the model at both the micro and macro level. It’s as important as any other element of risk or statistical analysis, or indeed the model building itself. At the distribution level historical data helps to both parameterise the distributions and in fact select them in the first place. As a minimum a few data points will help you to understand possible central tendencies and variability for your risks, and also generate a list of feasible distributions to choose from. With a reasonable number of observations @RISK for Excel can be used to fit distributions to the data taking care of both distribution selection and parameterisation simultaneously. Only five data points are technically needed, but a reasonable fit will require either more than that or other holistic information to achieve validity.

At the macro level total project cost estimates are often ignored from the portfolio perspective. Commonly high percentiles are reported from such models to use in a ‘contingency’ calculation, such as the P90 or P95. Whilst a high percentile, the P90 (say) should still be exceeded 10% of the time! If your projects never go over this percentile then either there are some major mitigating factors not included in the model or the volatility is being consistently overstated. Likewise, the P10 for total cost (these ‘good’ percentiles are rarely if ever reported or considered in project cost estimation work) should be bettered in roughly 10% of projects. If this is not the case then the upside risk has been overstated. This may be due to misconceptions about the positive skewing present in most cost/delay risks or mistakes made in the parameterisation of the risks where the estimate (“most likely” etc.) is actually the “best case” or close to it, rather than a central tendency of the process over time. There could also be other possibilities.

No matter how you look at it, the collection and intelligent use of historical data is integral to effective and useful risk analysis and management, and critical to achieving valid Monte Carlo simulation results. If you aren’t currently recording everything you can get your hands on start right now!

 

» Part 1
» Part 2



Rishi Prabhakar
Trainer/Consultant

Free Live Webcast this Thursday: Simulating the U.S. Economy: Where will we be in 100 years?

Monday, January 25, 2010 by DMUU Training Team
This Thursday, 28 January 2010 at 11am ET, Dr. William Strauss, President of FutureMetrics, will present a free live webcast entitled, "Simulating the U.S. Economy: Where will we be in 100 years?" Sign up now to attend the webcast.

There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable. 

To see where this growth imperative will take us we first have to see how we go to where we are today. This free live webcast first models the 20th century. The model is both complex and simple. The basic schematic of the model’s relationships is easy to understand. Furthermore, the core of the model is a simple production function that combines capital, labor, and the useful work derived from energy to generate the output of the economy. Complexity is contained in the solutions to the internal workings of the model. What is unique is that there are no exogenous economic variables.  Once the equations’ parameters are calibrated, setting the key outputs to “one” in 1900 results in their time paths very closely predicting the U.S. GDP and its key components from 1900 to 2006. 

The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? From 1900 to 2006 there are periods in which there was parameter switching. (The optimal parameters and the years for the switching were found using a constrained optimization technique.) That suggests that in the future there will also be changes. The experiment uses @RISK’s features (risk analysis software using Monte Carlo techniques) to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization using Monte Carlo simulation) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

----
William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics).

» Register now for this FREE live webcast
» View archived webcasts

Cost-Benefit Feedback Loop

Friday, January 15, 2010 by Holly Bailey
An anonymous comment in the Vail (Colorado) Daily News about the dangers of overanalyzing a decision reminded me that, while the benefits of risk analysis have been much vaunted, the costs of decision evaluation have not been clearly defined.  Sure, it's pretty easy to come up with a figure for a DFSS training effort or a budget for an entire risk management department. But what about the statistical analysis process itself?  

Well, there's staff time or your own time (which is worth something), Monte Carlo software, some portion of your computing costs,data acquisition, and on and on. Many variables. But the kind of costs I'm thinking of are the kind you rack up while you're analyzing, say, option valuation, and not doing something else.  These are opportunity costs.  They are what really limit how thoroughgoing your risk analysis becomes, which layer you drill down to--and they are very difficult to quantify.

How do you calculate whether the time you're spending in risk assessment is cost-effective? It's a problem of operations risk.  So I suppose you could enumerate all the other activities that would consume the same amount of time and model their paybacks.  But that would cost you more time in statistical analysis. . . . and you would be left in a positive feedback loop.

In the days ahead I'll be talking to risk management and operations research folks to find out how they decide how much analysis is just the right amount--not too much, and not too little.   I'll be surprised if I turn up any computational approaches--but who knows?  

Data Issues Part 1

Tuesday, January 12, 2010 by DMUU Training Team
In a recent public training workshop (for @RISK for Excel) I was reminded of an unusual fact regarding data.

Commonly @RISK for Excel is used to fit distributions to historical data for use in risk modelling, and it sure beats wildly guessing obscure parameters. However there are (naturally) a litany of woe-inducing problems with all historical data sets: non-stationary data series, extreme values/outliers, data recording errors, seasonality and heteroskedasticity to name a few. Excessive ‘cleansing’ of the data set is commonly prescribed, but the statistician in me cringes to even type those words! Quality control and transforming the data will help to eliminate most of those problems, but what about outliers?

In the early Naughties I was working for a large Australian bank, forecasting their daily call centre volumes for the purpose of planning staff levels and predicting service levels. A particular call centre averaged 30,000 calls per weekday. Yet on September 12th, 2001, calls dropped to less than 10,000. Along with the rest of the world, Australians were watching the terrorist attacks on television and the internet rather than calling to fix spelling mistakes in their contact details or transfer small sums of money between accounts. But what to do with that data point? Presuming the forecasting model is not intended to include such extreme events as terrorist attacks then the point could simply be filtered out of the data set and not thought of again.

But now consider a process that should include rarer events, such as flood damage or operational risk, as one of the risks in a model. If you have 10 years of good data (say), but the set includes an event that should only occur every 100 years. This level of impact is thus drastically overrepresented in the data and any fitted distribution will be biased toward such extremes. Yet the data point can not be completely ignored as such values can occur and the simulation models must have the capacity to sample such values (though with a reasonable likelihood). In this case the artistry that is fitting distributions to data comes to the fore. The data point could be removed from the set but not from our decision making process.

From the range of distributions that can be selected, the optimal choice should not only represent the remaining data well but also have a tail that samples events in the vicinity of those that have been excluded from the analysis with reasonable probability. No, that’s not always easy to do. But as with many elements of probabilistic modelling it simply must be done in order to provide useful information to decision makers.

Thus the context of the modelling can go a long way to determine the most appropriate steps to take with your data set. If that sounds like a subjective guideline then you read it correctly. Not enough people realise just how important experience and intuition can be in the seemingly prescriptive fields of mathematics and statistics. Fitting distributions to data is no different.

And yet that isn’t the unusual fact I was reminded of in the workshop! But I’ll leave that for Part 2 of my Data Issues blog.

Rishi Prabhakar
Trainer/Consultant

A Downturn for the Better

Thursday, January 7, 2010 by Holly Bailey
Honoring a time-honored tradition for the turn of the year, I've been looking back over the year just past to do a little retrospective trend-spotting.  Here's one that took me by surprise: in spite of the downturn in the economy, there was also a downturn in online fraud. It's counterintuitive--historically, hard times are correlated with rising crime--but apparently true.
 
Late last year, DigitalTransactions, an online publication catering to businesses engaged in the "electronic exchange of value," reported that the results of a survey of principals in these businesses showed an overall decline in fraud of about 1 percent.
 
The survey, sponsored and carried out annually by a California risk management company, is the first in its eleven-year history to show a fraud rate this low.  In 2009 North American merchants were expected to lose (a mere) $3.3 billion, in contrast to their loss of $4.0 billion in 2008.  
 
What's behind this good-news downturn?  Probably not increased honesty.  There was no data on attempted fraud, and the assumption is that the increased use of automated fraud detection tools cut the merchant's losses. The level of sophistication of these tools has ratcheted up to the level where neural network classification, risk analysis, and statistical analysis of correlated data can take place in real time during the processing of a transaction.  Furthermore, the combination of operational risk software with device identification of the purchaser's computer now make it difficult for a single computer to mob an online merchant with multiple bogus orders.

So the good news is not about improvements in human nature.  It's about improving the defenses of this booming sector of the economy.  

The role of software in risk management

Thursday, January 7, 2010 by DMUU Training Team
Today there is a heightened appetite for risk management due to global economic circumstances. But risk management has always been an intrinsic aspect of business to a higher or lesser degree. However, in the current technology-led business environment, the use of software to effectively manage risk makes logical sense. It provides a level of sophistication that the traditional processes simply cannot offer. Let me explain why.

Risk management essentially involves three stages – identification, quantification, and the on-going management of risks. In reality, these stages are not completely distinct from each other, with each stage influencing and informing the others. For example, an initial quantification of risks may lead to the conclusion that some of the identified risks are in fact not serious enough to warrant further consideration, or that the original description of the risk was not sufficiently precise for meaningful risk management measures to be put in place.

Each of these stages can benefit from the use of supporting risk modeling software. For instance, Microsoft Excel can be used to create a risk register, i.e. a database that records the risks identified, the assessment of the likelihood and impact of each of these risks, the mitigating actions that have been planned, and the assignment of responsibilities for these actions. However, there are many other software tools available, each designed for a specific purpose and focus. To illustrate, enterprise-wide risk management software focuses on the creation of integrated and holistic risk management systems, whereas Monte Carlo simulation and decision tree software place their emphasis on enhancing the quantitative analysis of risks.

The selection of the appropriate risk analysis software should involve very careful thought. The right decision can lead to a very effective implementation, whereas the wrong decision may result in a large amount of wasted investment.

There are some key considerations to bear in mind when selecting the risk modeling software. Choosing software based on how many staff will genuinely be required for the day-to-day risk management process is crucial. It is easy to select software based on the ideal situation that there will be a wide staff involvement in the risk management process. In reality, this may not be possible, potentially resulting in a cumbersome and inflexible solution being chosen over a more stand-alone and flexible application.

Similarly, knowing the level of risk quantification required is important. In fact, best practice risk management now involves the use of quantitative techniques, often using Monte Carlo simulation. When correctly conducted, the process of quantifying risks is rigorous and structured, can expose hidden or biased assumptions, as well as provide a more solid rationale upon which to base the major decisions.

Finally, determining the extent of on-going risk management needed for your business can assist with software selection. 

Needless to say, any software application will be most successful when used by appropriately trained and motivated staff, and when used as a supporting tool within an overall risk management process. Software is not a replacement for process.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis