@RISK Quick Tips: Automating @RISK risk analysis software with VBA

Tuesday, August 31, 2010 by DMUU Training Team
Presented at the Palisade Risk & Decision Analysis Conference New York City
Chris Albright, author of the book VBA for Modelers, presented a number of examples of how to automate @RISK, RISKOptimizer, and StatTools in Excel using Excel's VBA and Palisade's built-in object-oriented Excel Developer Kit. These examples include production applications, scheduling applications, World Series simulation, and more. All examples include macros written by Dr. Albright, so you'll need to enable macros when you open them.

» Download the examples
» Order Dr. Albright's book "VBA for Modelers"

Free Webcast This Thursday: The Use of the DecisionTools Suite in Biotechnology Project and Portfolio Decision Making

Monday, August 30, 2010 by DMUU Training Team
Vertex Pharmaceuticals, Inc. is a global biotechnology company based out of Cambridge, MA. The Company's strategy is to commercialize its products both independently and in collaboration with major pharmaceutical companies. Vertex's product pipeline is focused on viral diseases, cystic fibrosis, inflammation, autoimmune diseases, cancer, and pain.

Given the uncertainty of outcomes in the biotech industry, consideration of variability is an inherent part of the decision process. Often, the mean (average) is not a relevant decision criteria. This is especially true for smaller biotech companies like Vertex – the opportunity costs are extremely high because scarce capital resources would be invested elsewhere, with a higher probability of realistic return. For example, a company may reject a project which is profitable on average (positive Net Present Value) because some of the possible outcomes are unacceptable to the decision maker. Consideration of variability allows a decision maker to bring in their own risk tolerance into the decision. A similar argument applies when estimating a safety margin above a base case (e.g. in cost budgeting).

Vertex’s strategy and analytics group within the corporate finance division seeks to provide the senior management with dynamic revenue and profit forecasting methodology that helps to identify types of drugs that should be developed given a finite amount of cash and resources. A traditional financial view allows the user to identify scenarios and potential outcomes, but lacks the ability to show the range of potential values within each and every outcome. Vertex’s team uses the DecisonTools Suite to establish the average outcome, the variability of outcomes and to pressure-test risk and uncertainty of a particular scenario throughout the decision process.

Vertex’s team built a complex financial risk analysis model using @RISK to enhance its portfolio process. Monte Carlo simulation and optimization are used to analyze and optimize project and portfolio decisions, given short and long-term corporate strategy. @RISK is also frequently used throughout the business development process: simulating across multiple sales forecasts provides BD team with a range of potential outcomes, making it easy to pinpoint a particular scenario on a curve, along with its probability and value. TopRank turns the sensitivity analysis into a quick and seamless exercise, answering multiple what-if questions within minutes. Franchise and program leaders can now see a dollar effect of their program being delayed or advanced, adding supplementary indications to the development plan and even addressing the price uncertainties all at the same time. The simple interface of PrecisionTree along with tornado chart outputs makes it easy to explain the effect and importance of a particular assumption / decision to an audience with no finance background.

As the company continues to grow, adding more drugs and collaborations to its development pipeline, we will see in this free live webcast how the DecisionsTools Suite remains one of Vertex’s analytical tools of choice to enhance and guide the decision making process.

» Register now (FREE)
» View archived webcasts

Market decline versus speed to market – ‘A bird in the hand...’

Wednesday, August 25, 2010 by DMUU Training Team
I recently saw an interesting @RISK cashflow model from the portable phone industry. It modeled the uncertainty in the length and decline of overall market demand for a particular technology against five strategies for getting various application products to market as soon as possible. 

Using @RISK’s Simtable function, combined with Excel’s Index function, it was possible to run multiple simulations and see which strategy could take best advantage of the potential market, given the uncertainties in the development process, the possibility of competitors, the market take-up and the margins that might be achieved.

As is often the case in all aspects of life, the simulation revealed that ‘a bird in the hand is better than two in the bush’; it’s very comforting to know that @RISK risk analysis solutions can cut through loads of detail and come back with an answer that echoes received wisdom!

Ian Wallace, ACMA
Palisade Training Team

@RISK Quick Tips: Six Sigma Design of Experiments: Welding

Tuesday, August 17, 2010 by DMUU Training Team
A key application of @RISK is Six Sigma and quality analysis. This model demonstrates how @RISK can be used for a DOE analysis of a welding project.

Suppose you are analyzing a metallic burst cup manufactured by welding a disk onto a ring. You need to make sure weld strength is within safety limits. The model relates the weld strength to process and design factors, models the variation for each factor, and forecasts the product performance in relation to the engineering specifications. @RISK is used to model the variation in each factor, and simulate different outcomes for weld strength. Output includes After Six Sigma statistics for Cpk-Upper, Cpk-Lower, Cpk, and PPM Defects (or DPM). Standard @RISK statistical analysis functions (like RiskMean) are also used.

» Download the example: SixSigmaDOE.xls

Free Webcast This Thursday: “Assessing Your New Product, Process or Service Introduction Methodology: Is Yours Premier? Does it Enable Six Sigma Performance?”

Monday, August 16, 2010 by Steve Hunt
On Thursday, August 19, 2010, David Roy will present a free live webcast entitled. "Assessing Your New Product, Process or Service Introduction Methodology: Is Yours Premier? Does it Enable Six Sigma Performance? "

As companies make changes to or introduce new Products, Processes or Services, we observe a wide spectrum of methodology; from well defined process with trained resources, effective tools and excellent results - to no process, ad hoc application of tools, and frequent cycles of “Launch and Learn.”

Where does your methodology rank?

In this free live webcast, we will provide a framework for assessing the Process, People, Tools and Results for premier attributes in the New Product, Process, Services Introduction Methodology.


» Register now (FREE)
» View archived webcasts

Free Webcast This Thursday: “Assessing Your New Product, Process or Service Introduction Methodology: Is Yours Premier? Does it Enable Six Sigma Performance?”

Monday, August 16, 2010 by DMUU Training Team
On Thursday, August 19, 2010, David Roy will present a free live webcast entitled. "Assessing Your New Product, Process or Service Introduction Methodology: Is Yours Premier? Does it Enable Six Sigma Performance? "

As companies make changes to or introduce new Products, Processes or Services, we observe a wide spectrum of methodology; from well defined process with trained resources, effective tools and excellent results - to no process, ad hoc application of tools, and frequent cycles of “Launch and Learn.”

Where does your methodology rank?

In this free live webcast, we will provide a framework for assessing the Process, People, Tools and Results for premier attributes in the New Product, Process, Services Introduction Methodology.


» Register now (FREE)
» View archived webcasts

Oops! Didn’t see that coming! Part 4

Monday, August 9, 2010 by Steve Hunt

This is the conclusion of Dave Roy’s guest blog, we hope you have found them informative. Again, Dave comes to us from SSPI, Six Sigma Professionals, Inc., and taught Jack Welch and his entire staff their Six Sigma Green Belt training. Also, look for Dave’s free live webcast on August 19th, Assessing your New Product, Process or Service Introduction Methodology: Is yours premier? Does it enable Six Sigma performance?



Oops! Didn’t see that coming! Part 4
 

 

As a continuation from the July blog, we are now concluding with the “Optimize” and “Validate” phases of the ICOV (Identify-Conceptualize-Optimize-Validate) framework of a rigorous new design process as explained in “Services Design for Six Sigma – A Roadmap for Excellence”.

 

These phases are important because it allows time and methodology to optimize the design, develop all of the detailed documentation, verify performance and capability under operating conditions and manage an orderly transition to the new state.

 

The Optimize phase consists of a single stage (Design Optimization) and associated Tollgate 5 to validate successful completion of the requirements. 

 

The Design Optimization stage involves completing all of the detailed design documentation, building Prototypes of the design, simulating/analyzing Process Capability, preparing all Control Plans and updating the Design and Process Scorecards.

 

Tollgate 5 Exit Criteria:

o    Agreement that functionality and performance meet the customers’ and business requirements under the intended operating conditions.

o    Approval to proceed with the Validate stage.

 

Formal tools which can be used in this phase are Design Scorecard, Process Management, Mistake Proofing, Simulation, Change Management, Control Plans, Reliability and Robustness.

 

The Validate phase consists of two stages (Verification and Launch Readiness) and associated Tollgates (6 and 7) to validate successful completion of the requirements. 

 

The Verification stage involves developing Pilot plans, Piloting the new design and process and analyzing and making adjustments to achieve the desired functionality and performance under operating conditions.

 

Tollgate 6 Exit Criteria:

o    Agreement that functionality and performance from the pilot meet the customers’ and business requirements under the intended operating conditions.

o    Approval to proceed with the Launch Readiness stage.

 

Formal tools which can be used in this phase are Design Scorecard, Process Management, Mistake Proofing, Change Management, Control Plans, Statistical Process Control (SPC), and Confidence Analysis.

 

The Launch Readiness stage involves developing Pilot plans, Piloting the new design and process and analyzing and making adjustments to achieve the desired functionality and performance under operating conditions.

 

Tollgate 7 Exit Criteria:

o    Agreement that transition plans and training plans have been developed and are executable.

o    Approval to proceed with the Production stage.

 

Formal tools which can be used in this phase are Transition Plans, Training Plans, Process management, Change Management and Control Plans.

 

Following the ICOV model we have now used a formal methodology that allows us to validate performance at progressive economical stages and have improved the ability to detect unknown risks thus avoiding the Oops! Didn’t see that coming!. It should be mentioned that the methodology should be flexible and scalable to adjust for level of invention and risk. A brand new invention (Research & Development) that has never been deployed in similar conditions is much different than implementing a known solution (Application Engineering) under new conditions.

 » Part 1
 » Part 2
 » Part 3
 

 

 

BIO:

 

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. David holds an BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

 

Prediction Markets

Tuesday, August 3, 2010 by Holly Bailey
Although they've been around for the last 20 years or so, prediction markets have begun to make news for their application in business operations. Heralded early on in books like James Surowiecki's The Wisdom of Crowds, prediction markets are a fascinating alternative to traditional forecasting methods, such Monte Carlo simulation, which extrapolate future events from past patterns.  Essentially a betting exchange where participants stake something on the accuracy of the information they offer up, a prediction market is a way of capturing emerging patterns. 
 
Prediction markets can be public or closed private exchanges, as in most business applications. Here's how it might work: a business sets up an online portal to gather intelligence from its employees on such issues as scheduling or production costs.  Each employee has a limited number of points to wager with the information he or she offers, and these points are value-at-risk, which means that an employee is likely to offer only information that is accurate enough to be worth the points. 
 
Why bother to play at all?  Darwinian competition.  With each winning piece of information, the participant gains collective respect.  Maybe he or she advances in rank on a leader board or maybe the company honors its top participants in a ceremony. 
 
While the accuracy of prediction markets is still a topic of some fairly warm debate in applied mathematics, a number of risk analysis services are concentrating their solution portfolios on predictive markets.  

Tackling the energy crisis by managing demand, using risk modeling software

Thursday, July 29, 2010 by DMUU Training Team
One of the side-effects of the recession appeared to be a reduction in the demand for electricity as businesses and consumers alike looked to make savings on their outgoings. However, economic recovery seems to render this trend as temporary, meaning that the global need to tackle energy-consumption is as pressing as ever.

BC Hydro, Canada's third largest electrical utility provides an interesting case study in how to  ascertain the most effective ways to tackle the gap between supply and demand of electricity in British Columbia. Trends such as an expanding population growth, and the increase in energy usage per customer, have led to a rise in the demand for electricity across the region. By legislation, BC Hydro must aim to meet these energy needs through implementing cost-effective energy conservation approaches before it can turn to increasing the supply. 

The company has set itself one of the most aggressive targets in North America, with a plan to meet almost 75 percent of its incremental load through Demand Side Management (DSM) over the next 20 years. DSM projects include compact fluorescent light promotions; subsidies for energy efficient appliances; variable speed motor promotions (for home furnaces); and promotional activity aimed at motivating customers to use less energy.

BC Hydro uses @RISK risk analysis software to measure the uncertainty around its energy conservation efforts, both at the project stage and at a higher portfolio level. Around 60 projects were analysed on a case-by-case basis, and a probability distribution around the forecast outcome was developed. @RISK helps BC Hydro to capture the level of uncertainty of the estimated savings for each individual DSM venture.

In recognition that projects do not operate in isolation, BC Hydro also uses @RISK to explore the interrelationships between key uncertainties: the participation and savings per participant, and the participation across projects. The analysis showed that if a 'conservation culture' was developed in the province, it would result in an increase in energy savings across all programmes. However, it also illustrated that, if this culture failed to materialise, the performance of all programmes will be dragged down.

Exploring uncertainty using @RISK allowed BC Hydro to find the best balance between the uncertainties of supply side resources and those of relying heavily on energy conservation. Employing Decision Making Under Uncertainty helps BC Hydro to meet both financial risk analysis and environmental risk analysis goals. As a result, it expects to meet the majority of its incremental load growth through conservation measures.

» Read the BC Hydro case study

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

A Little Limelight

Tuesday, July 27, 2010 by Holly Bailey
Limelight--and by this I mean positively glowing publicity-- shines only occasionally on quantitative analysis, and rarely on Monte Carlo simulation.  But there was, 6 years ago, Michael Lewis's Moneyball, which established a place for statistical analysis in major league baseball.  Now there is Relativity Media, LLC, currently one of the heaviest hitting movie production companies in the business, and, more specifically, there is Ryan Kavanaugh, its CEO, and Ramon Wilson, its executive vice-president of business development.
 
Two things about Kavanaugh and Wilson make them unusual: they are leading a movie production firm that is not only alive but growing, and they use quantitative analysis for lots of decision making under uncertainty.  What can be more uncertain than investing in a movie? Only somewhat unusual for the movie business is the fact that these two decision makers are under thirty-five--it's a youth oriented business--and maybe this is correlated with their emphasis on making decisions by the numbers.  
 
"You can't think of it as money," Kavanaugh has been quoted as saying.  "You have to think of it as math."  Given the multimillion-dollar budgets Relativity underwrites--the raw size of the risks involved--it's probably more comfortable for everybody at Relativity to think math.  The kind of math Kavanaugh is particularly devoted to is Monte Carlo simulation, and he talks quite openly about his company's use of it.  When it comes to variables, he names names: principal actor,  genre, director, release date, PG  or R, although in all probability (sorry), each of these variables is probably a set of variables.  
 
"Everything has to run on the principle of profit.  We'll never let creative decisions rule our business decisions.  If it doesn't fit the model, it doesn't get done."  That doesn't mean, he has explained, that if he really likes a project, he and Wilson can't juggle the variables to make the film project fit the model.  They change the parameters to reveal the path to profit.  And profit he has--the estimated assets of Relativity are about $2 billion.  

So Kavanaugh qualifies as a mogul, a math-for-movies mogul.  When the spotlight falls on him, Monte Carlo simulation isn't far out of it.

  

Oops! Didn’t see that coming! Part 3

Monday, July 19, 2010 by Steve Hunt

We are pleased to welcome back to my blog consultant and trainer David Roy from Six Sigma Professionals, Inc.

 

 

Oops! Didn’t see that coming! Part 3
 

 

As a continuation from the June blog, we are now covering the “Conceptualize” phase of the ICOV framework of a rigorous new design process as explained in “Services Design for Six Sigma – A Roadmap for Excellence”.

 

This phase is important because it conceives, evaluates and selects good design solutions through robust process methodology which ensures alignment to the customer and the business needs.

 

Many design solutions skip this phase and become typically named as “Launch and Learn”.

 

The Conceptualize phase consists of two stages and associated Tollgates to validate successful completion of the requirements. 

 

The Concept Development stage involves translating Customer requirements into solution free Functional requirements, developing the System Level Conceptual Design, generating Concepts for required functions, Concept selection and translation of the Functional Requirements to Design Parameters.Click to Enlarge

An example of a Functional Requirement for a Customer Want of “Speedy Service” could be “Speed of Service” and a Design Parameter could be “Waiting Time

 

Tollgate 3 Exit Criteria:

  • Assessment that the Conceptual Development Plan & Cost will satisfy the customer base
  • A Decision the design represents an economic opportunity (if appropriate)
  • Verification adequate funding will be available to perform Preliminary Design
  • Identification of the Tollgate Keeper & the appropriate staff
  • An action plan to continue flow-down of the design Functional Requirements

 

The Preliminary Design stage involves creating the design documentation and configuration management, performing design analysis and testing, translating the Design Parameters into Process Variables and formulating the Production strategy.

An example of further mapping the Design Parameter of “Waiting Time” to a Process Variable could be “Number of Phone Lines

 

Tollgate 4 Exit Criteria:

  • Acceptance of the selected Solution/Design
  • Agreement the Design is likely to satisfy all Design Requirements
  • Agreement to proceed with the next stage of the selected Solution/Design
  • An action plan to finish the flow-down of the design Functional Requirements to design parameters and process variables

 

Formal tools which can be used in this phase are QFD, TRIZ/Axiomatic design, Measurement System Analysis (MSA), Failure Mode effect Analysis (FMEA), Design scorecard, Process mapping, Process management, Pugh Concept Selection, Robust Design, Design Scorecards, Design for X and Design reviews.

 

The next and final blog will cover the Optimize and Validate phases.

 

BIO:

 

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. David holds a BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

 


 » Part 1
 » Part 2


@RISK Quick Tips: Oil & Gas: Production and Economic Forecast using Exponential Decline.

Tuesday, July 13, 2010 by DMUU Training Team
@RISK has many applications for oil and gas exploration and production. This quantitative risk analysis model forecasts production, revenues, and present value based on exponential decline. Uncertain input factors include yearly production, decline rate, GOR, price of gas, price of oil, and rate of increase in oil and gas prices.

A SimTable function is also used in the Discount Rate input that is used to calculate Total NPV. This contains two possible values for Discount Rate – 12% and 14% - enabling you to run two back-to-back simulations to compare the effect of different discount rates on your Total NPV.

» Download the example: Declin.xls

Customised Solutions Using @RISK and VBA for Excel

Thursday, July 8, 2010 by DMUU Training Team
If you missed Palisade trainer Rishi Prabhakar's webcast "Customised Solutions Using @RISK and VBA for Excel," you can still view it in our archive.

The hour-long presentation explores the use of VBA for Microsoft Excel to control @RISK functionality, to simplify the process of risk analysis for resource-strapped businesses. Rishi explains the advantages (and limitations) of macro control for modelling and running simulations.

Simple examples are worked through to show the XDK (@RISK’s automation library) in action, from generic examples to a cost estimation model. This addresses elements of model construction, various simulation settings and finally reporting. The emphasis is on exposing the viewer to the various possibilities the XDK lends to the user rather than an in-depth VBA for Excel coding session.

Rishi Prabhakar holds a BSc in Mathematics from the University of Technology, Sydney Australia. Rishi has experience in the resources, infrastructure and primary industries, telecommunications, scientific research, banking and finance with an emphasis on operational risk.

With technical skills in the areas of modelling, simulation, statistical analysis, cost estimation, time series forecasting, customised solutions utilising VBA for Excel, and extreme value theory, Rishi has provided training and consulting services in risk and decision analysis for Palisade’s Asia Pacific office since 2005.


» Customised Solutions Using @RISK and VBA for Excel
» Webcast archive

Ensuring water supply when the heavens rarely open, using risk simulation software

Wednesday, July 7, 2010 by DMUU Training Team
Abu DhabiThe UK finally seems to be heading into summer after an unusually long and cold winter.  However, despite the amount of rain that falls on our 'green and pleasant land' (to the extent of major flooding on occasions), one of the anomalies of the UK weather system is that any prolonged warm period seems to be accompanied by the underlying threat of a hosepipe ban.

This is in stark contrast to many regions around the world that, despite seeing far less precipitation, manage a robust water supply.  Abu Dhabi for example has no rain for several months of the year, and relies almost completely on desalinated seawater for its potable water requirements.  The desalination process is challenging in terms of operation, costs, and environmental impact.  Whilst over-production capacity is expensive, at the other end of the scale it is essential that Abu Dhabi has sufficient water production capacity to support the Abu Dhabi government development plan (Abu Dhabi Plan 2030). 

This plan means that the Abu Dhabi Water & Electricity Company (ADWEC) is required by the Regulation and Supervision Bureau (RSB), the regulatory body of Abu Dhabi, to use a risk-based methodology to assess the water demand and required capacity.  As a result ADWEC uses @RISK risk analysis software to help it to forecast as accurately as possible the demand for water and electricity across the Emirate in order to plan for the optimum expansion as well as the efficient and effective use of water production plants.

@RISK enables ADWEC to model all feasible uncertainties in the variables that determine the quantity of water required over specific timescales, such as per capita water consumption rates and the rate of population growth.  The variables input into the @RISK risk simulation software model are based on the water demand categories such as domestic, agricultural and industrial. Factors with inherent uncertainties that affect the demand forecast outcome and must be modelled include: seasonal variation, distillers' unplanned outages, water losses, population growth rates and demand for housing.

By undertaking risk analysis of the variables involved in assessing demand and supply, ADWEC minimises the potential for water production capacity to be over or under deployed.  As a result of using @RISK to assist with its forecasting, planning and management strategies, ADWEC has been able to consistently meet with almost complete accuracy the Abu Dhabi Emirate water demand forecasts.

A useful lesson...

» Read the full ADWEC case study

Craig Ferri
EMEA Managing Director of Risk & Decision Analysis

Oops! Didn’t see that coming! Part 2

Tuesday, June 15, 2010 by Steve Hunt

Guest blogger David Roy Six Sigma Professionals, Inc., and taught Jack Welch and his entire staff their Six Sigma Green Belt training. Dave also has a quick survey for your input on structuring DFSS training. brings us the second installment of his four-part blog. Dave comes to us from SSPI,

 

--Steve Hunt

 
Oops! Didn’t see that coming! Part 2

We’d like to ask for your guidance by completing a short marketing survey to help SSPI structure our training in a way that is most useful to our community. This 8 question survey should take less than 5 minutes, and is anonymous. Your opinions are greatly appreciated.

As a continuation from the May blog, we are now covering the “Identify” phase of the ICOV framework of a rigorous new design process.

This phase is important because it establishes the framework for the concept, establishes the level of rigor required for the project management process, estimates the development cost, collects the Customer and Business requirements and the criteria for success.

 

The level of project management needs to be flexible and scalable depending on the Level of Effort (cost) and the Level of Innovation (risk) of the new concept.

 

Surely a project that will take a month to develop and has been done elsewhere requires less rigor that a concept that will take 3 years to develop and represents a brand new invention which has never been done before.

 

The I phase consists of two Tollgates during which an objective steering committee will decide whether to refine the work in the current phase, proceed or cancel the project. 

 

Tollgate 1 Exit Criteria are:

o     Decision To Collect The Voice Of The Customer To Define Customer Needs, Wants And Delights

o     Verification adequate funding is available to define Customer Needs

o     Identification of the Tollgate Keepers1 leader & the appropriate staff

 

Tollgate 2 Exit Criteria is successful demonstration of:

o     Assessment of market opportunity

o     Command a reasonable price or be affordable

o     Commitment to development of the Conceptual Designs

o     Verification adequate funding is available to develop the Conceptual Design

o     Identification of the Gate Keepers leader (gate approver) & the appropriate staff

o     Continue flow down of CTSs to Functional Requirements

Click to Enlarge 

Formal tools which can be used in this phase are Market/Customer research tools, Product Roadmaps, Process Roadmaps, Technology Roadmaps, Multigenerational plans, Quality Functional Deployment (House of Quality).

 

Market/Customer research tools may include Customer Relationship Management (CRM) Data, Surveys, Focus Groups, Conjoint Analysis and Kano Model Analysis.

 

The next blog will cover the Conceptualize phase

 

 

 

BIO:

 

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. Dave’s experience includes Product and Transactional so his examples are of interest to all. David holds an BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

» Part 1

Clear Legal Precedent for Dealing with Uncertainty

Monday, June 14, 2010 by Holly Bailey
A recent U.S. Court of Appeals case is timely not only because it involves corporate liability for ocean pollution when everybody in this country is morbidly tracking the BP spill in the Gulf but because it is a case in which the judge highlights and corrects some common misconceptions about Monte Carlo simulation.
 
In a consolidated case involving hazardous waste dumping in the Houston Ship Channel, the codefendants, Tenneco and Occidental, acknowledged liability for the  pollution cleanup, but they appealed a lower court's decision partly on the basis of the court's method of allocating costs. The court had called an environmental engineer as expert witness and statistical analyst.  The engineer used Monte Carlo software and court-established inputs for his model. The defendants challenged the court's inputs in the risk analysis model, and the Circuit Court decision rebutted their objections in clear terms.
 
Writing for the Fifth Circuit Court of Appeals, Judge Patrick Higginbotham said, "Monte Carlo measures the probability of various outcomes, within the bounds of input variables; to calculate Occidental's waste volume,. . .  Instead of simply averaging the input values, Monte Carlo analysis uses randomly-generated data points to increase accuracy, and then looks to the results that those data points generate. The methodology is particularly useful when reaching an exact numerical result is impossible or infeasible and the data provide a known range—a minimum and a maximum, for example—but leave the exact answer uncertain."
 
Responding to the charge that this method of statistical analysis is unreliable and untestable, Higginbotham responded,". . .the cited cases at most stand for the proposition that Monte Carlo analysis is unreliable when injected with faulty inputs, but nothing more. . . .  Monte Carlo simulation is not inherently untestable. . . . If anything, Monte Carlo provides greater certainty than the basic alternatives: using one of the three data or using the arithmetic average of all three."
 
Countering the challenge that the model results were "equivocal" the judge continued, " The Monte Carlo analysis—though it produced a statistical range of likely outcomes and not one determinative answer—supports choosing one result over another, and certainly assisted the district court in its decisionmaking."
 
The decisions-by-the-numbers guys certainly had their day in court.  The free advertising wasn't bad either.

(Data) Cleanliness Is Next To Godliness

Monday, June 7, 2010 by Steve Hunt

I’m pleased to welcome Palisade Six Sigma Partner Edward Biernat of Consulting with Impact as featured guest blogger. As well as running a successful consultancy, Ed is a noted Six Sigma educator and author.

 

--Steve Hunt

 

 

(Data) Cleanliness Is Next To Godliness

 

I recently had dinner with Eric Alden, a Master Black Belt for Xerox corporation.  Eric had just gotten back from the American Society for Quality’s  (ASQ) headquarters in Milwaukee where he was one of 200 Master Black Belts worldwide that generated the questions for the upcoming ASQ Master Black Belt certification examination (more on that in an upcoming post).  Eric had also recently completed a mini-course for the local ASQ chapter on data integrity.  We shared some war stories and came up with some common threads regarding data integrity.

 

1.       Just because it is a number doesn’t mean it is worth anything.  People get enamored with tons of data from process instrumentation, shop floor collection sources or Excel spreadsheets.  There seems to be a false security with this pile of data, and managers often look to the Black Belt to ‘sort it out’, because with all that data, the answer is in there somewhere.  Many a belt has crashed on the rocky reefs of bad data, often after tons of time and effort (and credibility) were wasted generating false answers.

2.       GIGO.  The Garbage In – Garbage Out philosophy of computing applies especially to existing corporate databases.  Here a few recent examples of GIGO.

a.       A belt wanted to analyze the specific timing of events in shop floor process and had tons of data from the process instrumentation that had times down to the fraction of a second.  After lengthy analysis, they found a significant difference between two shifts and forced the lesser shift to adopt the sequence of the more uniform shift.  After introducing costly production problems and actually hurting the overall process, the sensors were found to be faulty and the overall process subject to human manipulation to generate the ‘pretty charts’ that everyone expected.

b.      Office areas are not immune.  Something as simple as a checksheet to gather data to analyze when a particular computer error occurred can be in question, especially when the clerk fills in the times at the end of the shift from memory rather than logging the event as it occurs.

3.       Good data in bad spreadsheets.  Even if you get good data, having an inexperienced person setting up the spreadsheet can cause problems.  It is analogous to a person using a word processing software and making a table using spaces and tabs.  It looks like a great table until you have to manipulate it.  Then it falls apart.  Problems like merged cells, subtotals, random formula inserted in cells, etc. can make a Belt weep and cause significant errors in the resulting analyses.

4.       Useless manipulation.  Often a big issue is that management wants data sliced a certain way for no good reason.  This sometimes leads to the proliferation of additional spreadsheets or databases that needlessly add to complexity.  (Note: If you have an ERP system like Oracle or SAP, USE IT!  They are designed to house data and protect its integrity.  Plus the data entry screens typically allow for better and more accurate entry.  Few things are more wasteful than entering everything in the ERP system then re-entering it into a spreadsheet to appease a manager’s inability to adapt and change.)

 

What are some tactics for resolving these issues?

1.       On a macro level, start ensuring that the data that your company is collecting is sound data as part of the preparation for a Six Sigma launch, or a part of plain old good business.  Bad data slows down or stops a Six Sigma project dead in its tracks, changing it from getting something done to fixing the data. 

a.       Know catalog your data databases, including the extra ones (Excel, Access) that are usually relied upon but undocumented.

b.      Prioritize the data sources by synchronizing them with your Six Sigma launch sequencing. 

c.       Sample the data to insure its usefulness.  If it is bad, fix it.  This will give teams better data to start off with and will allow time for that data to accumulate for analysis.

2.       For specific projects, conduct a Measurement System Analysis (MSA) on you data sources (This tool is often used in the Measure phase of the DMAIC model).  We often think of MSA’s when it comes to physical measurements.  It is just as critical in the ‘softer’ data. 

a.       Pull the correct sample size.  In StatTools, under  Statistical Inference there is a Sample Size Selection tool that can be used to pull the correct amount of data needed for the analysis.

b.      Pull your data randomly and follow the trail to the actual entry point.  That may mean watching how individuals enter data, probing for special circumstances, etc.

c.       In your analysis, look for random factors such as vacation fill-ins.  Both Eric and I both had several experiences where one person was filling in for someone who is out sick or on vacation and, usually do to inadequate training, varies from the expected process.

3.       Pivot Tables are our friends.  Start today upgrading the skill sets of the people that do the actual data entry and first level analysis.  Train them in how to use tools like Picot Tables that slice the data but leave the actual spreadsheet intact.  The fewer merged cells, etc. that we fight with, the better.

4.       Managers – Trust your Belt.  If they say the data is bad, it probably is.  No matter how much you want an answer today, you may not be able to get one.  The good news is that some processes can be modeled using @RISK to begin improvement that is directionally correct while waiting for the data to compile.  Then the better data can be used to either update or replace the early model.

5.       Go hunting.  Find extraneous datasets and merge them / kill them.  The fewer that are out there, the more likely you will be able to ensure the integrity of those that remain.

 

Remember that data analysis is a funnel.  Tons of data leads to bunches of information which then can help us make some decisions.  Throwing bad data into the system is similar to throwing bad tomatoes into the food distribution system.  The end results can be pretty messy and difficult to clean up. 

 

Also, don’t miss Ed Biernat’s free live webcast DMAIC and Using a Non-Intuition Approach, Thursday, 11AM Eastern Time.

 

Sign up here:

https://palisade.webex.com/palisade/onstage/g.php?d=719996370&t=a

 

 

BIO:

 

Edward Biernat is the president of Consulting With Impact, Ltd., a training, coaching, and consultancy located in Canandaigua, NY that he founded in 1998.

Another take on the BP Oil Spill

Friday, May 28, 2010 by Steve Hunt

We are pleased to introduce you to consultant and trainer Sandi Claudell, today’s featured guest blogger. Sandi is CEO of MindSpring Coaching, and has been a valued Palisade Six Sigma Partner for quite some time. She is a Six Sigma Master Black Belt (Motorola), and is a Lean Master (Toyota Motors - Japan) among other notable achievements.

--Steve Hunt


Part 1: The Platform Disaster

Much has been said about the disastrous BP oil spill in New Orleans. If we use the theory of probability and reliability then have too many different companies responsible for a very complex construction and operation added to the chance of failure.

 

There is probably a cultural issue at work where each entity wanted to give the other what they wanted to hear rather than the truth. (For historic and recent examples: NASA Challenger and recent Toyota Prius problems). When we lose sight of quality and reliability of parts, construction, maintenance, testing under ALL conditions rather than the obvious few, etc. then we run high risks of failure. When you build 100+ wells and avoided disasters  . . . perhaps people fool themselves into thinking there never WILL be a disaster. They don’t look at a model that demonstrates the longer you go without such an event (given the input factors of how each element can and will fail) the closer you come to the event we all want to avoid.

 

They may or may not have used an integrated Systems Design  . . . not simply an engineering system but the system on how individuals work together, communicate with each other, act as a conforming unit or a more self-directed autonomous unit looking for and generating solutions outside the box. A team that is innovative and willing to look at all the possibilities and create a breakthrough design that was / is more mistake proof.

 

If they had used DFSS (Design for Six Sigma) then their designs would be more robust taking into consideration all the necessary safety precautions for human life as well as immediate response to a potential failure. As part of DFSS we use a statistical tool call Design of Experiments (Strategy of Formulations, Central Composites, etc.) where we can try very complex interactions (factors) with minimal effort / cost and maximum statistical accuracy. DoE creates prediction equations that allow us to model and ask questions of what would happen under different conditions. More importantly we can look at many different quality metrics (responses, outcomes, etc.) with the same experimental trial. If we replicate the test then we can even forecast what elements cause variation (very hard to detect in highly complex systems without the use of statistics).

 

If they had used an FMEA (Failure Mode Effect Analysis  . . . a tool used in Six Sigma) then they could have anticipated failures and put error proofing devices in place to detect and/or respond to potential faults BEFORE it is irreversible. If we add a Monte Carlo simulation to potential working conditions then the model forecasts probability plots and identifies key factors that will be critical to success or failure.

 

Perhaps they did indeed use a Monte Carlo using Crystal Ball. It is a good product but if they used Palisade’s @RISK and added some of the other tools provided by Palisade such as RISK Optimizer, Neural Tools, etc. then they could have analyzed the system in other dimensions besides a simple Monte Carlo, thus uncovering weaknesses BEFORE designing and/or building the platform and well.

 

Part 2: Capping the well head

 

In Lean there is a whole discipline called “Error Proofing Devices”. As part of the design effort we need to create first and foremost safety and other devices that prevent the error from occurring in the first place. If that line of defense fails then there should be devices built into the process designed to cap the well if your error proofing fails. If that line of defense fails then there should be a disaster response plan created and practiced and tested to ensure that the spill is repaired immediately.

 

Part 3: Treating the resulting spill

 

Again, Design of Experiments could test different materials, chemicals and methods to find the right combination to contain or otherwise manage the resulting oil spill. Trying one chemical only may be the age old definition of madness . . . trying the same thing over and over again expecting different results. Again, a robust design of experiments could aid in the process of finding a solution that is most effective and with multiple tests on the same samples ensure that is it the most safe for the environment and the population most directly in the path of the oil spill. These tests are ideally run years before such a spill however, doing something now is better than simply standing by and watching it happen.

 

Last but not least:

 

Management (Executives down to line managers) should have coaches. Coaches who can speak to the culture, the systems design, the tools and methods used in Lean Six Sigma and who can verify data analysis and help with the accurate interpretation of the data. These coaches should be independent . . . not a full time employee of the corporation as they are more likely to speak the truth and highlight risks as well as opportunities.

 

Now BP and all the other entities may have done some of what I mentioned above. But I would assume they must have left out one or more of the listed items or we wouldn’t be looking at the oil traveling into the wetlands around New Orleans right now. Hindsight is always brilliant but we can learn from our mistakes. We can create better cultures, systems, error proofing devices, Experimental Designs etc.

 

 

BIO:  

 

Sandi Claudell is CEO of MindSpring Coaching. She is a Master Black Belt in Six Sigma, a Lean Master and has worked as a consultant for many companies to initiate worldwide improvements. For more information or to contact Sandi please visit http://www.mindspringcoaching.com/.

Statistical Gizmos and the UK Election

Thursday, May 20, 2010 by Holly Bailey
The recent elections in the United Kingdom provided a really fun opportunity to see how extensively statistical decision evaluation and predictive modeling have penetrated popular culture.  The British press outdid themselves with online graphical gizmos that allowed readers to set the terms for outcome scenarios and let those spin out in true operations management style.
 
While The BBC offered an election seat calculator that really only translated voting percentages to number of Parliament seats won, the Guardian put up a Three-Way Swingometer.  With about 8, depending on which you count, parties in the fray, the Swingometer allowed readers to twiddle a dial to anticipate the effect of hypothetical party-shifting and election results.  
 
Next, Nate Silver, the election forecasting guru behind the FiveThirtyEight.com website, produced what he calls the Advanced Swingometer to offset the statistical disarray introduced by the original version's assumption of a uniform rate of "swing."  He backed this up with a demonstration of how elegant the statistical analysis  behind his model was. 
 
The Times came forward with a predictive map based on the predictions of gamblers in UK's lively betting shop scene.  Who know where those risk assessments came from.  
 
None of the online descriptions of the methods behind the gizmos were very detailed.  There were no mentions of named statistical analysis procedures, and this turns out to have been a good thing--because none of the gambits proved up to foreseeing the muddle that resulted from the actual voting.  If you wanted to try to come to a clear view of that, you will need to consult the decision tree posted by the BBC.