Project Risk Management Process

Free Webcast This Thursday: The Use of the DecisionTools Suite in Biotechnology Project and Portfolio Decision Making

Monday, August 30, 2010 by DMUU Training Team

Vertex Pharmaceuticals, Inc. is a global biotechnology company based out of Cambridge, MA. The Company's strategy is to commercialize its products both independently and in collaboration with major pharmaceutical companies. Vertex's product pipeline is focused on viral diseases, cystic fibrosis, inflammation, autoimmune diseases, cancer, and pain.

Given the uncertainty of outcomes in the biotech industry, consideration of variability is an inherent part of the decision process. Often, the mean (average) is not a relevant decision criteria. This is especially true for smaller biotech companies like Vertex – the opportunity costs are extremely high because scarce capital resources would be invested elsewhere, with a higher probability of realistic return. For example, a company may reject a project which is profitable on average (positive Net Present Value) because some of the possible outcomes are unacceptable to the decision maker. Consideration of variability allows a decision maker to bring in their own risk tolerance into the decision. A similar argument applies when estimating a safety margin above a base case (e.g. in cost budgeting).

Vertex’s strategy and analytics group within the corporate finance division seeks to provide the senior management with dynamic revenue and profit forecasting methodology that helps to identify types of drugs that should be developed given a finite amount of cash and resources. A traditional financial view allows the user to identify scenarios and potential outcomes, but lacks the ability to show the range of potential values within each and every outcome. Vertex’s team uses the DecisonTools Suite to establish the average outcome, the variability of outcomes and to pressure-test risk and uncertainty of a particular scenario throughout the decision process.

Vertex’s team built a complex financial risk analysis model using @RISK to enhance its portfolio process. Monte Carlo simulation and optimization are used to analyze and optimize project and portfolio decisions, given short and long-term corporate strategy. @RISK is also frequently used throughout the business development process: simulating across multiple sales forecasts provides BD team with a range of potential outcomes, making it easy to pinpoint a particular scenario on a curve, along with its probability and value. TopRank turns the sensitivity analysis into a quick and seamless exercise, answering multiple what-if questions within minutes. Franchise and program leaders can now see a dollar effect of their program being delayed or advanced, adding supplementary indications to the development plan and even addressing the price uncertainties all at the same time. The simple interface of PrecisionTree along with tornado chart outputs makes it easy to explain the effect and importance of a particular assumption / decision to an audience with no finance background.

As the company continues to grow, adding more drugs and collaborations to its development pipeline, we will see in this free live webcast how the DecisionsTools Suite remains one of Vertex’s analytical tools of choice to enhance and guide the decision making process.

» Register now (FREE)
» View archived webcasts

Oops! Didn’t see that coming! Part 2

Tuesday, June 15, 2010 by Steve Hunt

Guest blogger David Roy Six Sigma Professionals, Inc., and taught Jack Welch and his entire staff their Six Sigma Green Belt training. Dave also has a quick survey for your input on structuring DFSS training. brings us the second installment of his four-part blog. Dave comes to us from SSPI,

--Steve Hunt

Oops! Didn’t see that coming! Part 2

We’d like to ask for your guidance by completing a short marketing survey to help SSPI structure our training in a way that is most useful to our community. This 8 question survey should take less than 5 minutes, and is anonymous. Your opinions are greatly appreciated.

As a continuation from the May blog, we are now covering the “Identify” phase of the ICOV framework of a rigorous new design process.

This phase is important because it establishes the framework for the concept, establishes the level of rigor required for the project management process, estimates the development cost, collects the Customer and Business requirements and the criteria for success.

The level of project management needs to be flexible and scalable depending on the Level of Effort (cost) and the Level of Innovation (risk) of the new concept.

Surely a project that will take a month to develop and has been done elsewhere requires less rigor that a concept that will take 3 years to develop and represents a brand new invention which has never been done before.

The I phase consists of two Tollgates during which an objective steering committee will decide whether to refine the work in the current phase, proceed or cancel the project.

Tollgate 1 Exit Criteria are:

o Decision To Collect The Voice Of The Customer To Define Customer Needs, Wants And Delights

o Verification adequate funding is available to define Customer Needs

o Identification of the Tollgate Keepers1 leader & the appropriate staff

Tollgate 2 Exit Criteria is successful demonstration of:

o Assessment of market opportunity

o Command a reasonable price or be affordable

o Commitment to development of the Conceptual Designs

o Verification adequate funding is available to develop the Conceptual Design

o Identification of the Gate Keepers leader (gate approver) & the appropriate staff

o Continue flow down of CTSs to Functional Requirements

Formal tools which can be used in this phase are Market/Customer research tools, Product Roadmaps, Process Roadmaps, Technology Roadmaps, Multigenerational plans, Quality Functional Deployment (House of Quality).

Market/Customer research tools may include Customer Relationship Management (CRM) Data, Surveys, Focus Groups, Conjoint Analysis and Kano Model Analysis.

The next blog will cover the Conceptualize phase

BIO:

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. Dave’s experience includes Product and Transactional so his examples are of interest to all. David holds an BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

» Part 1

(Data) Cleanliness Is Next To Godliness

Monday, June 7, 2010 by Steve Hunt

I’m pleased to welcome Palisade Six Sigma Partner Edward Biernat of Consulting with Impact as featured guest blogger. As well as running a successful consultancy, Ed is a noted Six Sigma educator and author.

--Steve Hunt

(Data) Cleanliness Is Next To Godliness

I recently had dinner with Eric Alden, a Master Black Belt for Xerox corporation. Eric had just gotten back from the American Society for Quality’s (ASQ) headquarters in Milwaukee where he was one of 200 Master Black Belts worldwide that generated the questions for the upcoming ASQ Master Black Belt certification examination (more on that in an upcoming post). Eric had also recently completed a mini-course for the local ASQ chapter on data integrity. We shared some war stories and came up with some common threads regarding data integrity.

1. Just because it is a number doesn’t mean it is worth anything. People get enamored with tons of data from process instrumentation, shop floor collection sources or Excel spreadsheets. There seems to be a false security with this pile of data, and managers often look to the Black Belt to ‘sort it out’, because with all that data, the answer is in there somewhere. Many a belt has crashed on the rocky reefs of bad data, often after tons of time and effort (and credibility) were wasted generating false answers.

2. GIGO. The Garbage In – Garbage Out philosophy of computing applies especially to existing corporate databases. Here a few recent examples of GIGO.

a. A belt wanted to analyze the specific timing of events in shop floor process and had tons of data from the process instrumentation that had times down to the fraction of a second. After lengthy analysis, they found a significant difference between two shifts and forced the lesser shift to adopt the sequence of the more uniform shift. After introducing costly production problems and actually hurting the overall process, the sensors were found to be faulty and the overall process subject to human manipulation to generate the ‘pretty charts’ that everyone expected.

b. Office areas are not immune. Something as simple as a checksheet to gather data to analyze when a particular computer error occurred can be in question, especially when the clerk fills in the times at the end of the shift from memory rather than logging the event as it occurs.

3. Good data in bad spreadsheets. Even if you get good data, having an inexperienced person setting up the spreadsheet can cause problems. It is analogous to a person using a word processing software and making a table using spaces and tabs. It looks like a great table until you have to manipulate it. Then it falls apart. Problems like merged cells, subtotals, random formula inserted in cells, etc. can make a Belt weep and cause significant errors in the resulting analyses.

4. Useless manipulation. Often a big issue is that management wants data sliced a certain way for no good reason. This sometimes leads to the proliferation of additional spreadsheets or databases that needlessly add to complexity. (Note: If you have an ERP system like Oracle or SAP, USE IT! They are designed to house data and protect its integrity. Plus the data entry screens typically allow for better and more accurate entry. Few things are more wasteful than entering everything in the ERP system then re-entering it into a spreadsheet to appease a manager’s inability to adapt and change.)

What are some tactics for resolving these issues?

1. On a macro level, start ensuring that the data that your company is collecting is sound data as part of the preparation for a Six Sigma launch, or a part of plain old good business. Bad data slows down or stops a Six Sigma project dead in its tracks, changing it from getting something done to fixing the data.

a. Know catalog your data databases, including the extra ones (Excel, Access) that are usually relied upon but undocumented.

b. Prioritize the data sources by synchronizing them with your Six Sigma launch sequencing.

c. Sample the data to insure its usefulness. If it is bad, fix it. This will give teams better data to start off with and will allow time for that data to accumulate for analysis.

2. For specific projects, conduct a Measurement System Analysis (MSA) on you data sources (This tool is often used in the Measure phase of the DMAIC model). We often think of MSA’s when it comes to physical measurements. It is just as critical in the ‘softer’ data.

a. Pull the correct sample size. In StatTools, under Statistical Inference there is a Sample Size Selection tool that can be used to pull the correct amount of data needed for the analysis.

b. Pull your data randomly and follow the trail to the actual entry point. That may mean watching how individuals enter data, probing for special circumstances, etc.

c. In your analysis, look for random factors such as vacation fill-ins. Both Eric and I both had several experiences where one person was filling in for someone who is out sick or on vacation and, usually do to inadequate training, varies from the expected process.

3. Pivot Tables are our friends. Start today upgrading the skill sets of the people that do the actual data entry and first level analysis. Train them in how to use tools like Picot Tables that slice the data but leave the actual spreadsheet intact. The fewer merged cells, etc. that we fight with, the better.

4. Managers – Trust your Belt. If they say the data is bad, it probably is. No matter how much you want an answer today, you may not be able to get one. The good news is that some processes can be modeled using @RISK to begin improvement that is directionally correct while waiting for the data to compile. Then the better data can be used to either update or replace the early model.

5. Go hunting. Find extraneous datasets and merge them / kill them. The fewer that are out there, the more likely you will be able to ensure the integrity of those that remain.

Remember that data analysis is a funnel. Tons of data leads to bunches of information which then can help us make some decisions. Throwing bad data into the system is similar to throwing bad tomatoes into the food distribution system. The end results can be pretty messy and difficult to clean up.

Also, don’t miss Ed Biernat’s free live webcast DMAIC and Using a Non-Intuition Approach, Thursday, 11AM Eastern Time.

https://palisade.webex.com/palisade/onstage/g.php?d=719996370&t=a

BIO:

Edward Biernat is the president of Consulting With Impact, Ltd., a training, coaching, and consultancy located in Canandaigua, NY that he founded in 1998.

Oops! Didn’t see that coming!

Wednesday, May 12, 2010 by Steve Hunt

We are pleased to introduce you to consultant and trainer David Roy, our first guest blogger in my blog. Dave comes to us from SSPI, Six Sigma Professionals, Inc., and taught Jack Welch and his entire staff their Six Sigma Green Belt training. David’s blog will be the first in a series, and this initial entry also has a quick survey at the end for your input on structuring DFSS training.

--Steve Hunt

Oops! Didn’t see that coming!

How often do we hear these words after we have made a change to product, service or process?

We frequently solve one problem only to discover a new problem; or the solution we selected didn’t really resolve the problem.

There are many reasons for these surprises. Problem Solving sometimes addresses the symptoms and not the root cause. Useful solutions often have compromising harmful effects that we did not consider.

You may now be thinking; “Wow, if everything we do is going to turn out bad let’s not change anything.” The reality is that change is inevitable. Whether driven by rising customer expectations, innovative new technologies or even variation in inputs over time; change will occur.

Managing the design and implementation of these changes requires a more formal methodology than the prominent “Launch and Learn” method.

The sophistication of the methodology will vary depending on the magnitude of the risks associated with the change. If we are problem solving for variation in a standard process and trying to regain control simple tools such as Cause and Effect diagram and Failure Mode Effects Analysis and Standard Work may be all that is required.

When we start to explore reducing variation or introducing new technologies or process then we need to bring on a Design For Six Sigma (DFSS) methodology which incorporates elements such as Change Management, Robust Design, Reliability, Modeling & Simulation and Piloting & Prototyping.

Over the next 4 blogs we will cover the four phases of a DFSS project under the framework of I-dentify, C-onceptualize, O-ptimize, and V-erify or ICOV for short.

We will give a high level look at the steps within these phases and the tools used to reduce the risk of the change and un-intended consequences.

On another note, if you are able, we’d like to ask for your guidance by completing a short marketing survey to help SSPI structure our training in a way that is most useful to our community. This 8 question survey should take less than 5 minutes, and is anonymous. Your opinions are greatly appreciated.

http://www.surveymonkey.com/s.aspx?sm=2aQk8QF1eLB5MFQJC1pUXA_3d_3d.

BIO:

Put More Science into Cost Risk Analysis

Tuesday, May 4, 2010 by DMUU Training Team

At the 2010 Palisade Risk Conference in London, John Zhao of Statoil used a mock cost estimate contingency model to demonstrate how @RISK simulation functions can yield a more realistic project contingency through integrated qualitative risk assessment and quantitative risk analysis.

While future oil prices may be hard to predict due to low manageability, it is absolutely possible to scientifically forecast the sizes of risks that companies are willing to take, and such risks may include the probabilistic volumes of newly discovered reserves, the probability of meeting a project development schedule, chances of project cost overruns, and the likelihood of eroding entire project profitability. To achieve these goals, @RISK has lent a helping hand to business analysts for easier operation of complicated mathematical modelling.

Statoil, an international oil company, takes risk management seriously and has applied Monte Carlo simulation techniques in core and support businesses using @RISK. Such applications not only include the solo use of individual applications, but integrated combinations from drilling, reserve estimation, and well completion to cost and schedule controls at project execution. Besides the widespread uses of the software, Zhao discussed a specific application of @RISK to convincingly simulate required capital project contingency in detail.

A simplistic line-item ranging exercise using @RISK Monte Carlo simulation is no longer adequate to derive large capital project contingency, as empirical data confirmed that many disastrous cost overrunning projects were lack of contingency to cover the covert risks. In order to show management a complete risk picture on a project, both systemic risks (which empirical history has indicated a likelihood of occurring), and specific risks (which have discrete probabilistic characteristics), should be included in the overall project risk analysis. Therefore the combination of continuous PDF for project cost estimates, and discrete PDF for project risk registers, may prevail and provide management with a more convincing project cost contingency.

John Zhao is Quality and Risk Manager at StatoilHydro Canada Limited. He has 22 years project management experience in the petrochemical industry. He has authored many papers and made numerous presentations worldwide on the subject of risk and contingency management. In the past 10 years, John has developed his expertise in cost engineering and risk analysis for large downstream and oilsands upstream projects across Canada. His extensive knowledge in construction project qualitative risk assessment process has made him an expert on the subject in North America; his proprietary Monte Carlo model using @RISK is a popular tool for project contingency and escalation simulation. The quantitative model that John has built has integrated @RISK with PrecisionTree to help corporations conduct risk-based strategic decision-making.

» View the complete abstract and PDF presentation of "Put More Science into Cost Risk Analysis"
» Read Zhao's whitepaper, "Put More Science into Quantitative Risk Analysis"

The Economics of Supply Chain Risk Management using @RISK

Monday, May 3, 2010 by DMUU Training Team

At the 2010 Palisade Risk Conference in London, David Inbar of Minet Technologies presented a talk on supply chain risk management.

Supply chain risk management is an emerging field which has been growing significantly in importance because of modern management concepts such as lean, globalization and outsourcing. The mutual dependencies and close collaboration in modern supply chains create unique risks and challenges. Supply chain risk management is an economic process and choosing the elements and amount of risk mitigations should be based on economic measures.

Inbar's talk gave an overview of the concepts and process of supply chain risk management, and demonstrated how using Monte Carlo simulation techniques with @RISK risk analysis software adds value to the decision making under uncertainty processes and enables managers to purchase the most cost effective mitigations. Says Inbar, "An organization with the right risk management process can assure peace of mind to customers and supply chain partners."

David Inbar is the founder and managing director of Minet Technologies, a provider of professional services and technologies in supply chain and purchasing. Minet is active in the interfaces between business, processes and technologies in the world of supply chain and purchasing, creating methodologies and delivering projects and solutions.

» View a PDF of the presentation here
» Abstracts and presentations from the 2010 Palisade Risk Conference in London

Robust Risk Analysis for the Time/Expertise Poor – Part 2

Thursday, April 15, 2010 by DMUU Training Team

In my last blog I introduced the idea of a customised risk analysis solution to problems commonly faced in project risk management, especially cost estimation. Of course this idea is not uniquely applicable to project costs, but this paradigm is the simplest to explore, and that’s what I’m about to do.

Picture a risk register in a worksheet that has been created at a macro level to encapsulate most (all?) of the risks your projects may face. For any given project only a subset of these will be relevant – what is the best way to get these risks into a risk model on the next worksheet? By pressing a button of course! It is almost trivial to write code that picks up all selected risks and places them and the relevant data fields in the model worksheet. Sure beats manually copying and pasting individual line items and the transcribing errors that follow.

The next problem is utilising the workshopped parameters (likelihood of event, three-point estimates for severity etc.) in a logical way to be referenced by appropriate @RISK functions. Once a model structure has been agreed upon a macro button can place @RISK distributions where they ought to go, either logically due to the paradigm (using RiskBinomial, for example) or via a drop-down selection for dollar impact (RiskPert or RiskGamma, say). My clients have been especially thankful when I limit their choice of distribution and provide a simple flow-chart to follow to make this very decision. Reducing the propensity for arguments in risk workshops is worth its weight in gold; if we can assume that reducing this risk ‘weighs’ plenty!

Similarly one or two instances of the simulation settings are likely to satisfy all requirements, so these too can be activated by macro buttons. In this way a user can’t run a ‘poor’ simulation thus creating spurious results. The simulation output that is required can be placed into a report template attached to the model template and generated using yet another simply-labelled macro button. In this way there will be consistent reporting across the organisation allowing decision makers to become familiar and comfortable with simulation results they might otherwise ignore or be unaware of.

A risk model created by this process may not be the theoretically optimal one, but it will be valid and in context with its intended use. It will certainly be easy to use! The results will be consistent and should satisfy management’s desires as well as regulatory requirements.
The project cost estimation is but one example, and the above possibilities are far from the only ones imaginable. Additional complexity or alternate needs would be just as easily met simply with different code essentially without any practical limits. You don’t need to be an expert in Monte Carlo techniques and software to run robust, credible risk analyses. All you need is a risk analysis consultant who macro-controls the cumbersome and probabilistic elements, some appropriate simulation options and reporting procedures. Ask for me by name!

» Robust Risk Analysis for the Time/Expertise Poor – Part 1

Rishi Prabhakar
Trainer/Consultant

Robust Risk Analysis for the Time/Expertise Poor – Part 1

Tuesday, April 13, 2010 by DMUU Training Team

I have recently spoken to several clients whom have all came to the same conclusion about the risk analysis solution they think is most appropriate. They don’t want to do it, and I have no problem with that!

Of course that’s not precisely true. The benefits of Monte Carlo techniques in risk analysis are quite well understood and there is plenty of buy-in from businesses in the Australasian region. The trouble these businesses face (particularly in the realm of project cost estimation) is that the specific process of quantifying their risks for stochastic analysis and the ensuing simulation is not well understood and the means to ameliorate this appears to be beyond their reach. The modelling and simulation components of the project risk management process are not given adequate resources to be performed well, and certainly not to the extent that they provide the most useful information.

It is the case that many companies do not employ dedicated quantitative analysts. This means they have to rely upon some (maybe one) person in the team who has a non-zero quantity of experience and possibly training with risk simulation software to create a valid and credible stochastic model. This person is also not likely to be given enough time to do said task, thus the model inevitably suffers. It is my experience that most models – and all project cost estimation models – can be improved or actually need to be fixed.

So the corporate mind is willing, but the flesh is weak. How can this be addressed? No amount of additional training will suddenly allow you to overcome your time and resource constraints. Perhaps you can’t get the budget for training anyway or don’t want to master risk analysis software when it’s not really core to your role? The solution is one that I personally endorse (and provide!) as a risk analysis consultant – custom Excel programming.

VBA for Excel is a fairly simple language to learn, yet very powerful tool for automating repetitive or sometimes complex spreadsheet tasks. A customised solution involves writing VBA code to perform the tasks we’d rather not do ourselves in the risk analysis model. The “we” here refers to companies that find themselves in the situations previously described whereby they are incapable of creating and operating these models, not necessarily though any fault of their own. In my next blog I’ll examine some modelling problems/requirements and how they might be dealt with effectively using customisation.

Rishi Prabhakar
Trainer/Consultant

What Should You Get From a Simulation? Part 2

Wednesday, March 3, 2010 by DMUU Training Team

Where I left off last time was lamenting the use of Monte Carlo simulation to create a single value (statistic etc.) from a model. It might still not be clear why this is anathema to me, so here goes.

A simulation is not a number. It’s not one possible (future) outcome – that’s a scenario. Monte Carlo simulation is a methodology for understanding one’s exposure to outcomes not situated close to the central tendency of the process/project in question. Note the plural “outcomes”. Risk analysis, when done properly, should let you know essentially all possible outcomes and how likely they are for your model. Output from a simulation can include a plot of means (over time), or P5s, or P95s, or the mean ± one standard deviation or any number of statistics. But that’s not plotting a simulation! Let’s not give a minimalist graph too much credit.

Such statements also perpetuate the idea that simulation is only used for creating means (or other centrally tending statistics) and ignores the wealth of information available. Risk simulation software exists to help you do risk analysis which must include not only several statistics but also sensitivity information. It is all too easy to turn a risk assessment into a hunt for a regularly asked for percentile (such as the P90) and there ends the task. I see this a lot, especially in project cost estimation where the pressure both from management and regulatory bodies is to accurately estimate some large percentile. Once found there is usually scant further risk analysis.

Nothing good ensues. When risk analyses are run “to get ‘the’ number” they become simply another box to tick in a process and ultimately any benefits (perceived or actual) will be forgotten and lost to the ages. The notion of context is also lost. No single number by itself really means anything, or at least shouldn’t mean anything to a decision maker. I have often heard phrases like “the model returned/gave $1.2m” followed by an audience nodding in agreement. Huh? Which statistic are you talking about there, and how about reporting a few other numbers around it to place that $1.2m somewhere meaningful?

In the next installment I will look further into this issue of context and hopefully prove the necessity of an holistic approach to understanding and reporting simulation results.

» Part 1

Rishi Prabhakar
Trainer/Consultant

Data Issues Part 3

Tuesday, January 26, 2010 by DMUU Training Team

In Part 2 of this series I finished by asking what should be done with historical data, now that we have decided that storing it is probably a good idea. I won’t keep you waiting any longer.

Auditing and calibration of the model at both the micro and macro level. It’s as important as any other element of risk or statistical analysis, or indeed the model building itself. At the distribution level historical data helps to both parameterise the distributions and in fact select them in the first place. As a minimum a few data points will help you to understand possible central tendencies and variability for your risks, and also generate a list of feasible distributions to choose from. With a reasonable number of observations @RISK for Excel can be used to fit distributions to the data taking care of both distribution selection and parameterisation simultaneously. Only five data points are technically needed, but a reasonable fit will require either more than that or other holistic information to achieve validity.

At the macro level total project cost estimates are often ignored from the portfolio perspective. Commonly high percentiles are reported from such models to use in a ‘contingency’ calculation, such as the P90 or P95. Whilst a high percentile, the P90 (say) should still be exceeded 10% of the time! If your projects never go over this percentile then either there are some major mitigating factors not included in the model or the volatility is being consistently overstated. Likewise, the P10 for total cost (these ‘good’ percentiles are rarely if ever reported or considered in project cost estimation work) should be bettered in roughly 10% of projects. If this is not the case then the upside risk has been overstated. This may be due to misconceptions about the positive skewing present in most cost/delay risks or mistakes made in the parameterisation of the risks where the estimate (“most likely” etc.) is actually the “best case” or close to it, rather than a central tendency of the process over time. There could also be other possibilities.

No matter how you look at it, the collection and intelligent use of historical data is integral to effective and useful risk analysis and management, and critical to achieving valid Monte Carlo simulation results. If you aren’t currently recording everything you can get your hands on start right now!

» Part 1
» Part 2

Rishi Prabhakar
Trainer/Consultant

Adopting a healthy approach to risk

Tuesday, December 29, 2009 by DMUU Training Team

Having talked in previous posts as to why it’s important, and today how accessible it is for any size of organisation to adopt a healthy approach to risk, I’ll now take you through my top ten tips on how you can maximize your risk management programme:

1. Get buy-in
Risk management is not an optional extra. It is a business critical tool that is an asset and an integral part of the project. The company culture must be developed to embrace QRM (quantitative risk management) and DMU (decision making under uncertainty) in order that everyone understands their benefits and therefore accepts the need for them.

2. Get budget
Business tools cost money, but managing risk is an investment - not an overhead – and must be regarded as such. Allocating resource and making it a formal business process should be seen as an insurance policy. Not only will it help organisations make better decisions that will save them money in the long term but, by identifying potential risks and adverse events, it can protect them against unexpected costs in the future.

3. Get words
As with any organisational change, it is essential that everyone is clear on the new processes. Therefore a common risk language – or 'glossary' – needs to be developed to avoid misunderstanding and to ensure a consistent approach to QRM and DMU.

4. Get numbers
Qualitative assessment is essential, but numbers are more powerful – for example the percentage chance of meeting a deadline or budget. Monte Carlo simulation random sampling provides the margin of error for a venture and is a good way to illustrate the consequences of different courses of action. Risk management experts must ensure everyone understands these figures, and accepts them.

5. Get structure
Managing risk in order to make better-informed decisions requires an appropriate organisational structure. Individuals and groups need clearly defined roles, and must then each take responsibility for their own area of expertise.

6. Get lateral
Every organisation has risks that it deals with on a daily basis and which must therefore be factored in to the decision-making model. However, no enterprise operates in isolation, so other external variables must be included. For example, even a small rise in fuel costs could have a major effect on revenues if raw materials need transporting long distances.

7. Get perspective
Political, cultural and social risk factors can be explored by involving all stakeholders. Investing time and money in consultation and research ensures that businesses have a clear idea of the complete environment in which they operate, and therefore minimise the chances of products and services failing.

8. Get reporting
Risks, and the management of them, must be reviewed regularly – and the programme amended if necessary. This requires a regular reporting process, in which risks are clearly identified and prioritised.

9. Get with it
Being risk aware does not mean being risk averse. Businesses should guard against rigidly adhering to 'the way we've always done it' approach, instead keeping up-to-date, learning new tricks and not being afraid to be bold. Although risky on the surface, these tactics prevent being left behind – much of the potentially uncertainty can also be removed with QRM and DMU.

And finally…

10. Get it documented
Back up the commitment to a thorough QRM and DMU programme with documentation. This validates the budget and buy-in requested at the start. And it’s good for business – organisations this thorough are guaranteed a competitive edge.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Making Risk & Decision analysis accessible to all

Friday, December 18, 2009 by DMUU Training Team

It’s clear that the financial crisis has exposed a number of failings in the practice of risk management. In my last post I talked about the relevance risk analysis and the disciplines of ‘quantitative risk management’ (QRM) and ‘decision making under uncertainty’ (DMU) are to all sizes of organisations, be it large or small.

However, how accessible are these disciplines to the average size business across the globe today?

With the need to make more informed decisions more pressing by the day, thankfully QRM and DMU and now far more accessible than ever before. Traditionally systems tended to be expensive, enterprise-based applications targeted at large companies who were prepared to spend considerable time, money and human resources. The result was an all-singing, all-dancing product which often ended up underused due to confusion on the part of the very employees who were supposed to make it work.

Steady increases in computer processing have given the desktops of today as much power as the high-end servers of a few years ago, meaning that risk analysis and management is now an achievable goal for businesses of all sizes. Palisades @RISK and Decision Tools Suite software are such desktop risk and decision analysis tools – working within Microsoft Excel and therefore being accessible to a large number of users.

‘Monte Carlo Simulation’, a technique originally conceived by scientists working to develop the atomic bomb as part of the Manhattan Project, is an inherent part of @RISK, a cornerstone of the Suite. It enables users to introduce uncertainty into their previously static spreadsheets, which lets them look at things in a probabilistic, rather than a deterministic way. In layman’s terms, this means that rather than companies and individuals making decisions based on estimates or best guesses, they can see all the potential outcomes to a venture – and how likely these scenarios are to occur.

For many companies this significantly improves the decision-making process. Firstly it requires a change in the methodology of employees responsible for assessing risks and opportunities and secondly for the first time employees have a tool which allows them to communicate their recommendations to management or colleagues in a transparent and standardised way. Equally, being able to look at scheduling risk in a probabilistic and quantitative sense allows for the allocation of labour and resources in a way which minimises slack and wastage whilst maximizing ROI.

So, it would seem that the new ‘risk management’ language that is starting to develop in the workplace and being taught to a new generation of managers on MBA courses should be welcomed. With the accessibility of the technology available to assist them, we need to make sure that organisations do more than just pay lip service to QRM and DMU if they are to reap the rewards.

In my next blog I’ll be giving you the my top ten tips to adopting a health approach to risk, that will help businesses of all sizes maximize their risk management programmes.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Risk & Decision analysis – it’s not a dark art

Wednesday, December 16, 2009 by DMUU Training Team

The recent turbulence in the global economy has projected the word ‘risk’ into many everyday conversations, both commercial and personal: the unacceptable risks taken by fund managers which led to the collapse of major financial institutions; companies risking bankruptcy as a result of recession; the risk of people losing their jobs – and potentially their homes; and so on.

As a result there is also increased talk of risk analysis, which in turn has brought disciplines known as ‘quantitative risk management’ (QRM) and ‘decision making under uncertainty’ (DMU) firmly into the business zeitgeist. But for many small to mid-size companies, QRM and DMU are still regarded as something of a dark art and one that is not relevant to their day-to-day activities.

The truth is, that in boom or recession businesses make decisions every day – each with an associated level of risk. Much of this decision making is undertaken by looking into the issues facing a business, putting some numbers on them to calculate their impact, and then mitigating or allowing sufficiency contingency in the event that things go wrong.

Examining business decision-making in detail shows us that most businesses could benefit from making the link to risk analysis, and from there taking a more strategic approach to the discipline. Cost estimation, budgeting, cash flow forecasting, operational risk assessments, sales forecasting – in fact any part of a business where there is uncertainty can all be made more robust and meaningful.

Recession has brought the idea of QRM to the forefront of business owners’ minds. Essentially it is a valuable aide to making better, more informed decisions where the amount of uncertainty on which they are based is known.

Risk analysis is no longer a dark art, but in today’s economic climate, is an essential part of the business decision-making process, no matter what size the organisation.

In my next blog we’ll look what technology is available today that will help businesses making better decisions now and in the future.

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

KPMG Report Recommends Risk Management Expert, Stronger Risk Management

Tuesday, November 17, 2009 by DMUU Training Team

In a report issued last month, KPMG emphasizes the need for comprehensive, strategic risk management across an organization. Entitled “The Business Case for a Risk Executive: Leading Efforts to Avoid Surprises, Maneuver through Challenges, and Add Value,” the report notes that most current risk management efforts are specific to particular departments, projects, or regulations, and do not approach risk from an enterprise level. This had led to critical oversights and missed opportunities.

To address this gap, KMPG recommends the appointment of a risk executive. This person’s dedicated purpose is “to help prepare the organization to respond to change and the risks that emerge in changing times, and to turn those efforts into opportunities that benefit the organization.” More specifically, such an executive would unify risk approaches across business units and departments, standardize reporting, and establish a common risk “language.” (Note: Risk modeling software and Monte Carlo techniques play central roles in this effort.)

Expounding on the importance of risk management experts, the report concludes, "Without a risk executive, risk management efforts will likely continue to lag and hamper the organization’s effort to recover. But with a risk executive owning the process, risk management can move beyond a support role and help enable the organization to realize its strategic goals and rebuild business value."

» Read the full report (PDF)

Allocating Contingencies to Risk Events that were identified in a Risk Register

Friday, October 30, 2009 by DMUU Training Team

In a previous blog, I presented a very simple way to allocate contingencies to uncertain cost elements in the project risk management process. However, that methodology works well when there are not risk events that affect a cost element or a group of cost elements.
A risk event is described by two elements: the probability of occurrence and the conditional impact to the project given its occurrence. For example, we have a risk that describes the possibility of a new regulation. If it occurs, it will increment the cost of group of cost elements by a minimum of 10%, most likely 15%, and a maximum 20%. If the risk does not occur, no impact will be observed. Using a Discrete and a PERT distribution, we can model such risk such as:

When sampling from this distribution approximately only 20% of the time will generate a multiplier with a minimum of 1.1, most likely 1.5 and a maximum of 1.2; in 80% of instances the multiplier will be 1. That means that only 20% of the time the risk will increment the cost of selected cost elements by the multiplier previously described as show in the figure below:

In addition to risk events in our cost risk analysis models, we often use distributions that describe cost uncertainties. These distributions model ranges are mostly in a different order of magnitude. Therefore, the variance will also be in a even greater order of magnitude. For example, the cost of Item 3 modeled using a 3-point estimate (i.e., min 100,000, ML 120,000, and max 150,000) has a variance of 87,698,412.70), while the variance of the risk event is 0.0036.

If we are to distribute the contingency using the % of contribution of the variance method, the risk event that we just modeled will be ignored even though we know that such risk event has an impact that we cannot dismiss. Given this practical scenario, the method of variance contribution will not work appropriately.

As an alternative, we can use a tornado diagram that results from @RISK’s sensitivity analysis. Here we can use the regression coefficients to understand what risk events or uncertainties are affecting the total cost in a more drastic way. In the case that you also incorporated events that represent an opportunity to reduce cost, you will observe that the coefficient is negative; in your allocation calculations you should not consider negative coefficients.

In the figure below you can observe the Regression Tornado. Here risk events and uncertainties are represented in a scale that goes from 0 to +/-1:

Knowing the regression coefficient of each input that affects the total cost in a negative way, we can construct a table and obtain a normalized percent that can be used to distribute contingency. If for example, we have a contingency of $100,000, it can be distributed to each input proportionally to the regression coefficient as shown below.

Some risk management experts do not distribute the entire amount of the calculated contingency. It is common practice to distribute only a percentage of it (i.e., 70%). The remaining amount will be used as a reserve that will handle unidentified risks.

Javier Ordóñez, Ph.D
Director of Custom Solutions

Capitalizing Upon Market Inequities: A Game Plan for Successful Sports Wagering

Thursday, August 20, 2009 by DMUU Training Team

Clayton Graham is an adjunct professor of Statistics and Economics at DePaul University. He holds senior positions with the Chaos Group, Inc. and Analytical Advantages, LLC where he functions as a management consultant specializing in analytical and graphic econometrics.

He will present a case study at the 2009 the 2009 Palisade Conference: Risk Analysis, Applications, & Training. The conference is set to take place on 21 - 22 October at the Hyatt Regency in Jersey City, 10 minutes by PATH from Manhattan's Financial District.

See the abstract for Clay Graham's case study below, and see the full schedule for the Conference here.

Capitalizing Upon Market Inequities:
A Game Plan for Successful Sports Wagering

Sports wagering brings two separate "markets" together. First is the production market or the game itself. The second is the wagering or betting market. As a matter of practicality, the wagering market is itself in balance, i.e., bet clearing is covered through the process of adjusting the cost-payout ratio (the line). Betting lines are translated into an expected probability of winning. This resultant probability is frequently inconsistent with the probability of the team actually winning.

Hence, the opportunity to capitalize upon the dichotomy between the inequities of the production and gaming markets will be detailed and quantified. The presentation will include:

Fundamentals of gambling lines and odds,
Identification of key metrics,
Methods of production modeling baseball and basketball
(similarities and differences),
Integration of economics (investment) with production,
Economics of decision making.

Principal Palisade software utilized includes: StatTools (statistical analysis toolkit for Microsoft Excel), @RISK (risk analysis software using Monte Carlo techniques in Excel) and Evolver (genetic algorithm optimization in Excel). The presentation will have a heavy graphic and visualization emphasis. Theoretical statistics will be tightly tied with pragmatic realities of game modeling and economically based decision making.

Specific quantification will consist of:

Probabilities of winning a game,
Measurement bias of officials,
Quantification of player performance,
Expected values of return on investment,
Sports gambling optimizing algorithm.

Examples of actual results for current and past seasons along with predictions will be provided.

In short, it’s "Card Counting" for sports!

More information about this project can be seen at Baseballwon.com.

Risk Analysis Paralysis

Friday, August 7, 2009 by Holly Bailey

About a week ago, consultant James Yoakum posted a blog asking if the enormous amount of attention business pays to risk analysis has resulted in a state of "decision paralysis." Yoakum is a leadership coach who specializes in project risk management, operations risk, and analytical thinking.

What exactly is decision paralysis? According to Yoakum, it is a point in the decision process where the opportunity cost of further analysis is greater than its potential benefit. Here he's thinking in terms of decision evaluation involving too many variables, too much data, too many twists and turns, and--for him this is the killer--too great an emphasis on perfection. In short, unnecessarily complicated thinking. The result, he says, is too often a postponed decision, procrastination.

It's not hard to imagine a project management team swamped in just this kind of muddle. But how can that team figure out when good enough is good enough?

Doesn't this bring us back around full circle to apply risk assessment to the decision analysis process? And isn't this what a tautology is?

I'd love to hear from any of you who have experienced analysis paralysis and to have your thoughts on how to know when more analysis becomes too much analysis.

Monte Carlo Simulation: An Underutilized Six Sigma Tool

Wednesday, February 18, 2009 by Steve Hunt

Lean Six Sigma projects are performed in many areas of business. There are a few that require an estimation of future performance when there is no chance to test or evaluate the new process. On February 26, Rick Haynes of SmarterSolutions will share his expertise in a free live webcast that documents a case where a reliability testing effort provided a reliability model that needed to be extrapolated in order to estimate the total impact on warrantee costs.

The reliability model was developed through a logistic design of experiments. The resulting model was coded into an Excel spreadsheet and then modeled using @RISK to answer questions of future failure percentages. The results were used as inputs to focus on the need for proactive actions by the supplier in order to maintain a good customer experience. In the end, no additional actions were taken by the supplier and business continued with a manageable liability rather than with an unknown future risk

Thank you for your feedback on Six Sigma’s Thirst for Information blog posting. Working with industry experts to develop the webinar series and armed with voice of the customer that you supplied me, we’ve been able to ensure topics are of interest you. So please keep your thoughts coming.

Monte Carlo Simulation, It’s not just for DOE

Thursday, December 4, 2008 by Steve Hunt

One of the misperceptions about Monte Carlo Simulation is that it is only useful in reducing the number of “experiments” or “test runs” in conjunction with Design and Analysis of Experiments (DOE). Please don’t get me wrong it is a very powerful and useful tool for this use and is becoming more and more popular particularly during these economically challenging times.

What I’d like to touch on today, is using Monte Carlo software (@RISK) for a Lean project where you have virtually no data. When I was looking for my Lean Six Sigma Black Belt certification project, I contacted an aerospace company in the hopes they would have the need of process improvements in areas where there was much data so that I could apply all the wonderful tools I had learned about in my studies. To make a long story short, they had the need for process improvements in both manufacturing and engineering but their true needs were in the project management processes upstream of manufacturing. They regularly quoted a 12 month delivery time, but usually delivered in 17 months, with OTD rate of 22%. The only data we had to work with were, order date, date released to production and date the finished goods were shipped to the customer. The catch was, most orders were already late by the time manufacturing received all the parts to build the unit.

In order to identify the “problems”, we took one afternoon created a Process Flow Map by interviewing each of the departments and employees who were in the value stream, identified all the steps, assigned a best case, worse case and mostly likely time duration for each step, assigned distributions for each and created a Monte Carlo simulation model. The output of the model indicated a 20% OTD Rate and an average delivery rate of ~16.5 months. Not exact, but close enough to validate our model to allow the team to use @RISK’s sensitivity graphs to pinpoint which of the 80 plus process steps were contributing to the most variation. This also gave the team a starting point to hit the ground running. They are currently working on the suggested process improvements, which should save them ~$500k/year and reduce the average lead time to ~10 months.

If you would like more information, please let me know, I’d be happy to share more about the project with you.

Palisade Conference Provokes Six Sigma Buzz

Friday, November 21, 2008 by Steve Hunt

Last week, Palisade Corporation held its North American User Conference; it was a very successful event that brought together @RISK Users from around the world. Presentations and discussions touched on topics such as the subprime mortgage crisis, financial risk management, modeling flu, project risk management and of course, the ways to Monte Carlo simulation in Six Sigma.

It was great to see such a high level of interest in the Six Sigma related presentations and buzz they created in both the social networking opportunities as well as the feedback forms that were submitted after the conference. This shows despite the economic difficulties and the natural tendency to eliminate all unessential spending, Six Sigma and Design for Six Sigma is rightfully viewed as part of the solution.

SigmaFlow’s president Jay Holstine, presented Process Mapping for Knowledge Transfer: Doing More with Less. A very pertinent topic in today’s economic times, which will be presented live as an ISSSP Focused Session on November 25 at 2pm EST. Please join us.

Ed Biernat from Consulting with Impact led a presentation on the use of Six Sigma in Process Industries. If you are interested in viewing his presentation, Lean Six Sigma Applicatin of @RISK Part I, it can be viewed online. Part II will be live on December 12, 2008 at 1pm EST where he will dive deeply into the use of @RISK in this case study. Please join us.

A recent article, Executives Switch to Survival Mode, in the Wall Street Journal indicates that two of the top issues in crisis management can be managed with a strong Lean Six Sigma program, these were:

Excellence in Execution – Whether on the shop floor or in administrative processes, there is no longer room for inaccuracies or waste.
Speed, flexibility and adaptability to change is another area where a strong Six Sigma program mitigates the effects of crisis.

The interest at our User Conference in exploring the use of @RISK to reduce project cycle times and costs indicates to me that smart business leaders are looking to reduce risks and strengthen their companies during this time of crisis.