Genetic Algorithm Optimization

Free Webcast This Thursday: The Use of the DecisionTools Suite in Biotechnology Project and Portfolio Decision Making

Monday, August 30, 2010 by DMUU Training Team

Vertex Pharmaceuticals, Inc. is a global biotechnology company based out of Cambridge, MA. The Company's strategy is to commercialize its products both independently and in collaboration with major pharmaceutical companies. Vertex's product pipeline is focused on viral diseases, cystic fibrosis, inflammation, autoimmune diseases, cancer, and pain.

Given the uncertainty of outcomes in the biotech industry, consideration of variability is an inherent part of the decision process. Often, the mean (average) is not a relevant decision criteria. This is especially true for smaller biotech companies like Vertex – the opportunity costs are extremely high because scarce capital resources would be invested elsewhere, with a higher probability of realistic return. For example, a company may reject a project which is profitable on average (positive Net Present Value) because some of the possible outcomes are unacceptable to the decision maker. Consideration of variability allows a decision maker to bring in their own risk tolerance into the decision. A similar argument applies when estimating a safety margin above a base case (e.g. in cost budgeting).

Vertex’s strategy and analytics group within the corporate finance division seeks to provide the senior management with dynamic revenue and profit forecasting methodology that helps to identify types of drugs that should be developed given a finite amount of cash and resources. A traditional financial view allows the user to identify scenarios and potential outcomes, but lacks the ability to show the range of potential values within each and every outcome. Vertex’s team uses the DecisonTools Suite to establish the average outcome, the variability of outcomes and to pressure-test risk and uncertainty of a particular scenario throughout the decision process.

Vertex’s team built a complex financial risk analysis model using @RISK to enhance its portfolio process. Monte Carlo simulation and optimization are used to analyze and optimize project and portfolio decisions, given short and long-term corporate strategy. @RISK is also frequently used throughout the business development process: simulating across multiple sales forecasts provides BD team with a range of potential outcomes, making it easy to pinpoint a particular scenario on a curve, along with its probability and value. TopRank turns the sensitivity analysis into a quick and seamless exercise, answering multiple what-if questions within minutes. Franchise and program leaders can now see a dollar effect of their program being delayed or advanced, adding supplementary indications to the development plan and even addressing the price uncertainties all at the same time. The simple interface of PrecisionTree along with tornado chart outputs makes it easy to explain the effect and importance of a particular assumption / decision to an audience with no finance background.

As the company continues to grow, adding more drugs and collaborations to its development pipeline, we will see in this free live webcast how the DecisionsTools Suite remains one of Vertex’s analytical tools of choice to enhance and guide the decision making process.

» Register now (FREE)
» View archived webcasts

Oops! Didn’t see that coming! Part 4

Monday, August 9, 2010 by Steve Hunt

This is the conclusion of Dave Roy’s guest blog, we hope you have found them informative. Again, Dave comes to us from SSPI, Six Sigma Professionals, Inc., and taught Jack Welch and his entire staff their Six Sigma Green Belt training. Also, look for Dave’s free live webcast on August 19^th, Assessing your New Product, Process or Service Introduction Methodology: Is yours premier? Does it enable Six Sigma performance?

Oops! Didn’t see that coming! Part 4

As a continuation from the July blog, we are now concluding with the “Optimize” and “Validate” phases of the ICOV (Identify-Conceptualize-Optimize-Validate) framework of a rigorous new design process as explained in “Services Design for Six Sigma – A Roadmap for Excellence”.

These phases are important because it allows time and methodology to optimize the design, develop all of the detailed documentation, verify performance and capability under operating conditions and manage an orderly transition to the new state.

The Optimize phase consists of a single stage (Design Optimization) and associated Tollgate 5 to validate successful completion of the requirements.

The Design Optimization stage involves completing all of the detailed design documentation, building Prototypes of the design, simulating/analyzing Process Capability, preparing all Control Plans and updating the Design and Process Scorecards.

Tollgate 5 Exit Criteria:

o Agreement that functionality and performance meet the customers’ and business requirements under the intended operating conditions.

o Approval to proceed with the Validate stage.

Formal tools which can be used in this phase are Design Scorecard, Process Management, Mistake Proofing, Simulation, Change Management, Control Plans, Reliability and Robustness.

The Validate phase consists of two stages (Verification and Launch Readiness) and associated Tollgates (6 and 7) to validate successful completion of the requirements.

The Verification stage involves developing Pilot plans, Piloting the new design and process and analyzing and making adjustments to achieve the desired functionality and performance under operating conditions.

Tollgate 6 Exit Criteria:

o Agreement that functionality and performance from the pilot meet the customers’ and business requirements under the intended operating conditions.

o Approval to proceed with the Launch Readiness stage.

Formal tools which can be used in this phase are Design Scorecard, Process Management, Mistake Proofing, Change Management, Control Plans, Statistical Process Control (SPC), and Confidence Analysis.

The Launch Readiness stage involves developing Pilot plans, Piloting the new design and process and analyzing and making adjustments to achieve the desired functionality and performance under operating conditions.

Tollgate 7 Exit Criteria:

o Agreement that transition plans and training plans have been developed and are executable.

o Approval to proceed with the Production stage.

Formal tools which can be used in this phase are Transition Plans, Training Plans, Process management, Change Management and Control Plans.

Following the ICOV model we have now used a formal methodology that allows us to validate performance at progressive economical stages and have improved the ability to detect unknown risks thus avoiding the Oops! Didn’t see that coming!. It should be mentioned that the methodology should be flexible and scalable to adjust for level of invention and risk. A brand new invention (Research & Development) that has never been deployed in similar conditions is much different than implementing a known solution (Application Engineering) under new conditions.

» Part 1
» Part 2
» Part 3

BIO:

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. David holds an BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

Oops! Didn’t see that coming! Part 3

Monday, July 19, 2010 by Steve Hunt

We are pleased to welcome back to my blog consultant and trainer David Roy from Six Sigma Professionals, Inc.

Oops! Didn’t see that coming! Part 3

As a continuation from the June blog, we are now covering the “Conceptualize” phase of the ICOV framework of a rigorous new design process as explained in “Services Design for Six Sigma – A Roadmap for Excellence”.

This phase is important because it conceives, evaluates and selects good design solutions through robust process methodology which ensures alignment to the customer and the business needs.

Many design solutions skip this phase and become typically named as “Launch and Learn”.

The Conceptualize phase consists of two stages and associated Tollgates to validate successful completion of the requirements.

The Concept Development stage involves translating Customer requirements into solution free Functional requirements, developing the System Level Conceptual Design, generating Concepts for required functions, Concept selection and translation of the Functional Requirements to Design Parameters.

An example of a Functional Requirement for a Customer Want of “Speedy Service” could be “Speed of Service” and a Design Parameter could be “Waiting Time”

Tollgate 3 Exit Criteria:

Assessment that the Conceptual Development Plan & Cost will satisfy the customer base
A Decision the design represents an economic opportunity (if appropriate)
Verification adequate funding will be available to perform Preliminary Design
Identification of the Tollgate Keeper & the appropriate staff
An action plan to continue flow-down of the design Functional Requirements

The Preliminary Design stage involves creating the design documentation and configuration management, performing design analysis and testing, translating the Design Parameters into Process Variables and formulating the Production strategy.

An example of further mapping the Design Parameter of “Waiting Time” to a Process Variable could be “Number of Phone Lines”

Tollgate 4 Exit Criteria:

Acceptance of the selected Solution/Design
Agreement the Design is likely to satisfy all Design Requirements
Agreement to proceed with the next stage of the selected Solution/Design
An action plan to finish the flow-down of the design Functional Requirements to design parameters and process variables

Formal tools which can be used in this phase are QFD, TRIZ/Axiomatic design, Measurement System Analysis (MSA), Failure Mode effect Analysis (FMEA), Design scorecard, Process mapping, Process management, Pugh Concept Selection, Robust Design, Design Scorecards, Design for X and Design reviews.

The next and final blog will cover the Optimize and Validate phases.

BIO:

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. David holds a BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

» Part 1
» Part 2

Another take on the BP Oil Spill

Friday, May 28, 2010 by Steve Hunt

We are pleased to introduce you to consultant and trainer Sandi Claudell, today’s featured guest blogger. Sandi is CEO of MindSpring Coaching, and has been a valued Palisade Six Sigma Partner for quite some time. She is a Six Sigma Master Black Belt (Motorola), and is a Lean Master (Toyota Motors - Japan) among other notable achievements.

--Steve Hunt

Part 1: The Platform Disaster

Much has been said about the disastrous BP oil spill in New Orleans. If we use the theory of probability and reliability then have too many different companies responsible for a very complex construction and operation added to the chance of failure.

There is probably a cultural issue at work where each entity wanted to give the other what they wanted to hear rather than the truth. (For historic and recent examples: NASA Challenger and recent Toyota Prius problems). When we lose sight of quality and reliability of parts, construction, maintenance, testing under ALL conditions rather than the obvious few, etc. then we run high risks of failure. When you build 100+ wells and avoided disasters . . . perhaps people fool themselves into thinking there never WILL be a disaster. They don’t look at a model that demonstrates the longer you go without such an event (given the input factors of how each element can and will fail) the closer you come to the event we all want to avoid.

They may or may not have used an integrated Systems Design . . . not simply an engineering system but the system on how individuals work together, communicate with each other, act as a conforming unit or a more self-directed autonomous unit looking for and generating solutions outside the box. A team that is innovative and willing to look at all the possibilities and create a breakthrough design that was / is more mistake proof.

If they had used DFSS (Design for Six Sigma) then their designs would be more robust taking into consideration all the necessary safety precautions for human life as well as immediate response to a potential failure. As part of DFSS we use a statistical tool call Design of Experiments (Strategy of Formulations, Central Composites, etc.) where we can try very complex interactions (factors) with minimal effort / cost and maximum statistical accuracy. DoE creates prediction equations that allow us to model and ask questions of what would happen under different conditions. More importantly we can look at many different quality metrics (responses, outcomes, etc.) with the same experimental trial. If we replicate the test then we can even forecast what elements cause variation (very hard to detect in highly complex systems without the use of statistics).

If they had used an FMEA (Failure Mode Effect Analysis . . . a tool used in Six Sigma) then they could have anticipated failures and put error proofing devices in place to detect and/or respond to potential faults BEFORE it is irreversible. If we add a Monte Carlo simulation to potential working conditions then the model forecasts probability plots and identifies key factors that will be critical to success or failure.

Perhaps they did indeed use a Monte Carlo using Crystal Ball. It is a good product but if they used Palisade’s @RISK and added some of the other tools provided by Palisade such as RISK Optimizer, Neural Tools, etc. then they could have analyzed the system in other dimensions besides a simple Monte Carlo, thus uncovering weaknesses BEFORE designing and/or building the platform and well.

Part 2: Capping the well head

In Lean there is a whole discipline called “Error Proofing Devices”. As part of the design effort we need to create first and foremost safety and other devices that prevent the error from occurring in the first place. If that line of defense fails then there should be devices built into the process designed to cap the well if your error proofing fails. If that line of defense fails then there should be a disaster response plan created and practiced and tested to ensure that the spill is repaired immediately.

Part 3: Treating the resulting spill

Again, Design of Experiments could test different materials, chemicals and methods to find the right combination to contain or otherwise manage the resulting oil spill. Trying one chemical only may be the age old definition of madness . . . trying the same thing over and over again expecting different results. Again, a robust design of experiments could aid in the process of finding a solution that is most effective and with multiple tests on the same samples ensure that is it the most safe for the environment and the population most directly in the path of the oil spill. These tests are ideally run years before such a spill however, doing something now is better than simply standing by and watching it happen.

Last but not least:

Management (Executives down to line managers) should have coaches. Coaches who can speak to the culture, the systems design, the tools and methods used in Lean Six Sigma and who can verify data analysis and help with the accurate interpretation of the data. These coaches should be independent . . . not a full time employee of the corporation as they are more likely to speak the truth and highlight risks as well as opportunities.

Now BP and all the other entities may have done some of what I mentioned above. But I would assume they must have left out one or more of the listed items or we wouldn’t be looking at the oil traveling into the wetlands around New Orleans right now. Hindsight is always brilliant but we can learn from our mistakes. We can create better cultures, systems, error proofing devices, Experimental Designs etc.

BIO:

Sandi Claudell is CEO of MindSpring Coaching. She is a Master Black Belt in Six Sigma, a Lean Master and has worked as a consultant for many companies to initiate worldwide improvements. For more information or to contact Sandi please visit http://www.mindspringcoaching.com/.

Oops! Didn’t see that coming!

Wednesday, May 12, 2010 by Steve Hunt

We are pleased to introduce you to consultant and trainer David Roy, our first guest blogger in my blog. Dave comes to us from SSPI, Six Sigma Professionals, Inc., and taught Jack Welch and his entire staff their Six Sigma Green Belt training. David’s blog will be the first in a series, and this initial entry also has a quick survey at the end for your input on structuring DFSS training.

--Steve Hunt

Oops! Didn’t see that coming!

How often do we hear these words after we have made a change to product, service or process?

We frequently solve one problem only to discover a new problem; or the solution we selected didn’t really resolve the problem.

There are many reasons for these surprises. Problem Solving sometimes addresses the symptoms and not the root cause. Useful solutions often have compromising harmful effects that we did not consider.

You may now be thinking; “Wow, if everything we do is going to turn out bad let’s not change anything.” The reality is that change is inevitable. Whether driven by rising customer expectations, innovative new technologies or even variation in inputs over time; change will occur.

Managing the design and implementation of these changes requires a more formal methodology than the prominent “Launch and Learn” method.

The sophistication of the methodology will vary depending on the magnitude of the risks associated with the change. If we are problem solving for variation in a standard process and trying to regain control simple tools such as Cause and Effect diagram and Failure Mode Effects Analysis and Standard Work may be all that is required.

When we start to explore reducing variation or introducing new technologies or process then we need to bring on a Design For Six Sigma (DFSS) methodology which incorporates elements such as Change Management, Robust Design, Reliability, Modeling & Simulation and Piloting & Prototyping.

Over the next 4 blogs we will cover the four phases of a DFSS project under the framework of I-dentify, C-onceptualize, O-ptimize, and V-erify or ICOV for short.

We will give a high level look at the steps within these phases and the tools used to reduce the risk of the change and un-intended consequences.

On another note, if you are able, we’d like to ask for your guidance by completing a short marketing survey to help SSPI structure our training in a way that is most useful to our community. This 8 question survey should take less than 5 minutes, and is anonymous. Your opinions are greatly appreciated.

http://www.surveymonkey.com/s.aspx?sm=2aQk8QF1eLB5MFQJC1pUXA_3d_3d.

BIO:

David Roy is an integral part of the Six Sigma community. He taught GE’s Jack Welch and entire staff Six Sigma, as well as served as Senior Vice President of Textron Six Sigma. He is a Certified GE Master Black Belt, was instrumental in developing GE’s DMADV (DFSS) methodology, and has taught 3 waves of DFSS Black Belts. Dave’s experience includes Product and Transactional so his examples are of interest to all. David holds an BS in Mechanical Engineering from The University of New Hampshire. He is also the co-author “Services Design for Six Sigma – A Roadmap for Excellence”

Neural Nets Writ Small

Friday, May 7, 2010 by Holly Bailey

Of all the statistical analysis techniques I receive news alerts for, the neural network flashes up on my screen most often. While I, like many of you, really enjoy the big-screen futuristic applications of neural nets--prediction of sun storms is a splendid recent example--there is a quieter trend ramping up at a more down-to-earth level. The nano level,that is the itsy-bitsy, teeny-weeny, the molecular level.

For at least the past five years, the nanotechnology industry has been predicting and prototyping ways to incorporate neural networks into nano-machines. This innovation has proved to be very handy for sensing devices. The nano-sensor combines receptor particles with electronics controlled by a neural network algorithm. The neural net sorts through the sensor responses to uncover patterns that trigger alerts.

This year there was a flurry of media attention focused on one of these sensing technologies, the nano-nose, which uses an array of nano-receptors coordinated by a neural network. These sensors are being promoted to sniff out everything from explosives to disease.

One indication of the expected adoption of applications that combine nano with neural is the advertising for neural network algorithms that can downsize to nano. But more than one of the nano-machine innovators has commented on the need to develop more robust statistical analysis techniques to improve the accuracy of the sensors. Which means that there will be more neural network to shrink, which means that the algorithms advertised today may already be outdated.

Whatever the commercial considerations and no matter how blasé we become about technological possibility, there is still a big wow factor in packing a high-powered computing technique into such infinitesimal space, and you can be certain the nano people will be harnessing neural networks to many new kinds of more-mini-than-micro machines.

20 Questions in a New Orbit

Thursday, April 15, 2010 by Holly Bailey

An Ottawa toy developer is trying to make a jet-propelled leap from an online game to space travel. His vehicle? A neural network designed as the back end system for a game of 20 questions. Twelve years ago Robin Burgener wrote a neural net program to train on the sequences of player responses to questions--beginning with Animal? Vegetable? Mineral?--posed by the neural network,

The game is does more than pose simple yes-or-no answers to lead you to a conclusion. The neural network algorithm is able to pose different questions in different orders, and it gets the right answer about 80 percent of the time. ,

Now, apparently, the sky's the limit for Burgener's neural network. He was scheduled to make a presentation late last month at the Goddard Space Flight Centre explaining the potential uses for a neural networked 20 questions on board a space craft. These uses center broadly on troubleshooting technical and equipment problems and subsequently anticipating future problems.

If, as he claims is true, his neural net guessing program can work around responses that are misleading or downright lies, what that would mean for space travelers, he concludes, is that "if a sensor fails, you're able to see past it."

I know what he means, I think, but I myself don't tend to look past sensors.

Robust Risk Analysis for the Time/Expertise Poor – Part 2

Thursday, April 15, 2010 by DMUU Training Team

In my last blog I introduced the idea of a customised risk analysis solution to problems commonly faced in project risk management, especially cost estimation. Of course this idea is not uniquely applicable to project costs, but this paradigm is the simplest to explore, and that’s what I’m about to do.

Picture a risk register in a worksheet that has been created at a macro level to encapsulate most (all?) of the risks your projects may face. For any given project only a subset of these will be relevant – what is the best way to get these risks into a risk model on the next worksheet? By pressing a button of course! It is almost trivial to write code that picks up all selected risks and places them and the relevant data fields in the model worksheet. Sure beats manually copying and pasting individual line items and the transcribing errors that follow.

The next problem is utilising the workshopped parameters (likelihood of event, three-point estimates for severity etc.) in a logical way to be referenced by appropriate @RISK functions. Once a model structure has been agreed upon a macro button can place @RISK distributions where they ought to go, either logically due to the paradigm (using RiskBinomial, for example) or via a drop-down selection for dollar impact (RiskPert or RiskGamma, say). My clients have been especially thankful when I limit their choice of distribution and provide a simple flow-chart to follow to make this very decision. Reducing the propensity for arguments in risk workshops is worth its weight in gold; if we can assume that reducing this risk ‘weighs’ plenty!

Similarly one or two instances of the simulation settings are likely to satisfy all requirements, so these too can be activated by macro buttons. In this way a user can’t run a ‘poor’ simulation thus creating spurious results. The simulation output that is required can be placed into a report template attached to the model template and generated using yet another simply-labelled macro button. In this way there will be consistent reporting across the organisation allowing decision makers to become familiar and comfortable with simulation results they might otherwise ignore or be unaware of.

A risk model created by this process may not be the theoretically optimal one, but it will be valid and in context with its intended use. It will certainly be easy to use! The results will be consistent and should satisfy management’s desires as well as regulatory requirements.
The project cost estimation is but one example, and the above possibilities are far from the only ones imaginable. Additional complexity or alternate needs would be just as easily met simply with different code essentially without any practical limits. You don’t need to be an expert in Monte Carlo techniques and software to run robust, credible risk analyses. All you need is a risk analysis consultant who macro-controls the cumbersome and probabilistic elements, some appropriate simulation options and reporting procedures. Ask for me by name!

» Robust Risk Analysis for the Time/Expertise Poor – Part 1

Rishi Prabhakar
Trainer/Consultant

Making Optimal Choices, or Just Making Choices? Part 4

Tuesday, March 30, 2010 by DMUU Training Team

It has taken four entries but I’ll finish this blog stream now with a discussion on optimisation optimisation. That’s not a typo. It’s an art form that is analogous to elegant modelling, as opposed to ‘just’ modelling. The tipping competition model not only opened my eyes to the world of optimisation but also that not all models are created equally, even if they are numerically equivalent.

A few rounds into the season we were allowed to buy and sell riders, which was great if you’d bought some duds at the start! But from a modelling point of view the complexity had increased exponentially. There was now a time component to the model structure as well as different prices for the riders based on their performance to date. My first attempt to model this was quite cumbersome with dozens of 0/1 decision cells to indicate buying riders at the start of the season and then the buying and selling of riders after four rounds. While mathematically correct I wasn’t doing Evolver any favours by having such complex dependencies between so many decision cells. The optimisation was taking far too long to converge, so much so that the final solution when the optimisation was stopped after what I considered to be a reasonable length of time was greatly impacted by the initial solution.

Now this of course isn’t usually the way things work with Evolver; as a sophisticated genetic algorithm optimiser the global optimal solution should be found regardless of the initial conditions. However the time taken to get to such a solution can be extended greatly if optimising an unnecessarily convoluted model. After initially blaming the software I realised I could simplify the model by turning two decisions (“buy” and “sell”) into one (“change status”). This immediately removed one third of the decision variables and straight away the optimisation converged quickly to a global optimal solution regardless of the initial feasible solution. Evolver can only work with the model you give it!

The act of optimising the optimisation can be a subtly tricky one, but is necessary if you are to have confidence in your optimised solutions and thus the decisions made. Building a model that ‘works’ is only the first piece to the puzzle. If the solutions aren’t stable then can you really be sure you’re producing the best answer? No. And if the model is being used to decide which of the multi-million dollar projects you will proceed with (or some other equally critical decision) I’m sure you’d want to have some certainty around the answer provided by Evolver. Of course if you’d like some help with an optimisation model from experienced risk analysis consultants feel free to contact the consulting team at Palisade!

» Making Optimal Choices, Part 1
» Making Optimal Choices, Part 2
» Making Optimal Choices, Part 3

Rishi Prabhakar
Trainer/Consultant

Making Optimal Choices, or Just Making Choices? Part 3

Friday, March 26, 2010 by DMUU Training Team

Part 2 of this blog ended with me very quickly stating that the MotoGP tipping comp optimiser was identical in structure to a portfolio optimisation problem, where the portfolio could contain stock or other assets, or even projects. Let’s look at this in a little more detail as I’m sure you’re reading this to find how to optimise your own decisions rather than wondering how I went in the tipping competition!

In my model there was a fixed budget (though less could be spent if desired) to spend on riders, with the aim of maximising their total points haul. In the real world you may have a total budget of say $100m to invest in a range of projects perhaps many hundreds of millions of dollars in total value each of which have certain expected returns. At its simplest this decision evaluation will find the most (expected) profitable portfolio of the projects. This is an inclusion/exclusion grouping model, but it is very simple to optimise assets with a continuous level e.g. the amount of money invested in various shares etc. Another real example I have seen when working with an investment company here in Australia was a model whose goal was simply to find the portfolio mix that came closest to the total allowable spend without exceeding it.

Further realism can be included by using constraints should there be the need. A resource constraint may mean there has to be a limit to the number of projects that can be run simultaneously. There may also be a minimum number of projects determined by management as a mitigation strategy. Such constraints are very simple to employ using Evolver and add value to the decision analysis without the need to provide specific risk analysis/Monte Carlo simulation information for the model.

A slightly more sophisticated method of turning an optimisation into a useful portfolio risk management tool where uncertainty hasn’t been specifically modelled is to estimate the possible downside of each asset and include it in the calculation of the portfolio’s ‘score’. The Evolver software comes standard with over twenty example spreadsheets for your educational pleasure, of which “Portfolio Mix.xls” gives one method for doing just this.
In the next (and final) instalment of the Making Optimal Choices blog I will explore the idea that not all optimisations no matter how mathematically correct will produce the same results in good time, and that elegant modelling should always be the goal prior to firing up Evolver.

And so you know, I came second in the competition. Next year I’m hoping to go one better!

» Making Optimal Choices, Part 1
» Making Optimal Choices, Part 2

Rishi Prabhakar
Trainer/Consultant

Making Optimal Choices, or Just Making Choices? Part 2

Thursday, March 18, 2010 by DMUU Training Team

In my last blog entry I introduced the notion that optimal decision making wasn’t ‘on the radar’ for many clients in Australasia, and laid out a couple of ideas why. I too once focussed on Monte Carlo simulation rather than decision evaluation, but last year the most obscure event changed that.

Call me a nerd of you will, but I like modelling problems in Excel. There is skill involved in setting up a problem such that the model assumptions aren’t too gross, and an art to making the model elegant. This elegance can be very important to optimisation problems, but more on that later. My first homemade optimisation problem was generated by motorcycle racing! MotoGP, to be precise. A friendly tipping competition with friends was formed at the start of the 2009 season with the following structure:

Entrants played the role of Team Manager.
Team Managers had a fixed budget to spend on riders.
Either a few good riders could be purchased, or many lesser riders, or something in between.
The team that had accumulated the most points at the end of the season was the winner and received kudos!

Although the future results could not be known of course so I set up and ran the optimisation with Evolver after the event to see what the optimal team selection would have been. Historical data could have been used to discover the type of rider mix that tended to be optimal and thus make an informed decision for this competition. The risk in having only a few riders was that any misfortune would have a big negative impact on the points won, whereas a team consisting of many (cheaper) riders was less likely to suffer such a fate. This downside scenario will be modelled into the 2010 MotoGP Team Manager predictive, optimised model (currently in production)!

What has this to do with the corporate world? Replace “team” with portfolio and “riders” with “assets”, “shares” or “projects” and you have a classic portfolio optimisation model. I hadn’t created this model with business applications in mind but I realised that was precisely what I was doing. An instant later I realised just how useful Evolver would be in many decision scenarios even though it doesn’t incorporate uncertainty (RISKOptimizer does).

In the next instalment I will further explore some practical applications for Evolver and you’ll see just how universally appropriate it can be.

» Making Optimal Choices, Part 1

Rishi Prabhakar
Trainer/Consultant

Making Optimal Choices, or Just Making Choices? Part 1

Tuesday, March 16, 2010 by DMUU Training Team

Something has troubled me for some time regarding the choices being made in risk land. I train and work with many clients whom have adopted Monte Carlo simulation techniques (via @RISK for Excel) into the day-to-day running of their businesses. By doing so they (hopefully) now have a good understanding of the exposure they are facing be it in project cost estimation, discounted cash flow analysis or, well, anything really. But this is only one facet of risk and decision assessment, specifically dealing with the descriptive statistical output from a simulation. What of the decision evaluation component? Why aren’t more of my customers analysing the decisions they make, or better yet actually optimising them? I have a few ideas why.

If you’re in business you have to make decisions. Big ones, little ones, yes/no, multiple state and continuous value decisions. Decisions that impact other decisions in simple or complex dependency structures. But are you making the best decisions possible? I’m sure important decisions aren’t being made completely randomly (I hope!) but I see many companies who rely completely upon qualitative techniques for their decision making (experience, gut feel, etc.) which of course means optimality is no more than a hoped for outcome rather than something that is actually being worked towards.
Firstly the decision model must be identified and then quantified, and this can be a difficult task. There is a level of modelling aptitude necessary for effective modelling that goes beyond merely knowing Excel and its functions, and into the construction of logical mathematical descriptions of possibly complicated processes. Relevant decisions need to be identified and the impact of those decisions combined into a formula that can be mathematically optimised. A critical component to all this is the knowledge that spreadsheet models can actually be optimised, and that in cases where Excel’s Solver fails there are Palisade products (Evolver and RISKOptimzer) that can perform optimisations under virtually any circumstance.

I too used to focus on Monte Carlo simulation rather than decision evaluation, and this was mainly a product of the clients I was dealing with almost exclusively when I first worked for Palisade. In my next blog I’ll tell you why that changed and also get a little more into the nuts and bolts of optimisation.

Rishi Prabhakar
Trainer/Consultant

Free Webcast this Thursday: "Simulating the U.S. Economy: Where will we be in 100 years?"

Tuesday, January 26, 2010 by DMUU Training Team

There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable.

In this free webcast, Dr. William Strauss models the next 100 years, based on the last century's data. The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? The experiment uses @RISK’s risk analysis and Monte Carlo techniques to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization software) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

Palisade is pleased to host this presentation from Dr. William Strauss.

William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics). Read more of Dr. Strauss' bio here.

» Complete abstract of "Simulating the U.S. Economy: Where will we be in 100 years?"
» Register now (FREE)
» View archived webcasts

Free Live Webcast this Thursday: Simulating the U.S. Economy: Where will we be in 100 years?

Monday, January 25, 2010 by DMUU Training Team

This Thursday, 28 January 2010 at 11am ET, Dr. William Strauss, President of FutureMetrics, will present a free live webcast entitled, "Simulating the U.S. Economy: Where will we be in 100 years?" Sign up now to attend the webcast.

There is an assumption that drives all of our expectations for how our economy will be in the future. That assumption is one of endless economic growth. Clearly endless exponential growth is impossible. Yet that is what we base all of our expectations upon. We all agree that zero or negative economic growth is bad (just look around now at the effects of the Great Recession). But we also know logically that 2% or 4% annual growth every year leads to an exponential growth outcome that is unsustainable.

To see where this growth imperative will take us we first have to see how we go to where we are today. This free live webcast first models the 20th century. The model is both complex and simple. The basic schematic of the model’s relationships is easy to understand. Furthermore, the core of the model is a simple production function that combines capital, labor, and the useful work derived from energy to generate the output of the economy. Complexity is contained in the solutions to the internal workings of the model. What is unique is that there are no exogenous economic variables. Once the equations’ parameters are calibrated, setting the key outputs to “one” in 1900 results in their time paths very closely predicting the U.S. GDP and its key components from 1900 to 2006.

The experiment in this webcast is about the future. If the model can very closely replicate the last 100 years, what does it have to say about the next 100 years? From 1900 to 2006 there are periods in which there was parameter switching. (The optimal parameters and the years for the switching were found using a constrained optimization technique.) That suggests that in the future there will also be changes. The experiment uses @RISK’s features (risk analysis software using Monte Carlo techniques) to generate new combinations of parameters for each of tens of thousands of runs of the simulation. Changes in the parameters represent potential exogenous policy choices.

The “doing what you did gets you what you got” scenario leads to a surprising and unsettling outcome. The experiments using Evolver (genetic algorithm optimization using Monte Carlo simulation) do find a path that works. Obviously if it is not “business-as-usual” that leads to a stable outcome, it is some other way. The policy choices that lead to a stable outcome suggest that the future of capitalism is not going to be what we expect it to be.

----
William Strauss is the President and founder of FutureMetrics. He brings more than thirty years of strategic planning, project management, data analysis, and modeling experience into the company’s stock of knowledge capital. Bill’s professional history includes executive positions as director, president, and senior vice president, as well as positions as senior analyst and field coordinator. He has an MBA (specializing in Finance) and a PhD (Economics).

» Register now for this FREE live webcast
» View archived webcasts

Predicting Customer Will

Tuesday, January 12, 2010 by Holly Bailey

If hindsight is twenty-twenty, foresight--at least in the world of market research--still has a ways to go. Simulation, both with Monte Carlo software and with a conjoint simulation approach, has been used by market researchers for some time now. Recently David G. Bakken,who maintains a blog on the Smart Data Collective site, pointed out that the drawback of these models is that even those that incorporate random number generation are static. That is, the inputs and the coefficients determine the model outcomes.

What's wrong with deterministic models? Nothing, I gather, except for the limitation that those that are applied to marketing research questions tend to treat the target customers, the companies devising product strategies, and their affiliates in advertising and PR as blocs that make decisions without benefit of individual will.

Agent-based models, which were born in the social sciences, simulate the interactions of multiple players, each of whom will act, absolutely rationally, in his or her own best interests. Bakken believes that agent-based modeling used in tandem with traditional risk analysis models or evolutionary programming methods such as genetic algorithms, offers a more dynamic means of accounting for the future behavior of potential customers.

On the face of it, Bakken's proposal seems to have merit. If the technique works for the social sciences, maybe it will work for marketing research. After all, what is marketing if not a commercial application of social science?

Data Issues Part 1

Tuesday, January 12, 2010 by DMUU Training Team

In a recent public training workshop (for @RISK for Excel) I was reminded of an unusual fact regarding data.

Commonly @RISK for Excel is used to fit distributions to historical data for use in risk modelling, and it sure beats wildly guessing obscure parameters. However there are (naturally) a litany of woe-inducing problems with all historical data sets: non-stationary data series, extreme values/outliers, data recording errors, seasonality and heteroskedasticity to name a few. Excessive ‘cleansing’ of the data set is commonly prescribed, but the statistician in me cringes to even type those words! Quality control and transforming the data will help to eliminate most of those problems, but what about outliers?

In the early Naughties I was working for a large Australian bank, forecasting their daily call centre volumes for the purpose of planning staff levels and predicting service levels. A particular call centre averaged 30,000 calls per weekday. Yet on September 12th, 2001, calls dropped to less than 10,000. Along with the rest of the world, Australians were watching the terrorist attacks on television and the internet rather than calling to fix spelling mistakes in their contact details or transfer small sums of money between accounts. But what to do with that data point? Presuming the forecasting model is not intended to include such extreme events as terrorist attacks then the point could simply be filtered out of the data set and not thought of again.

But now consider a process that should include rarer events, such as flood damage or operational risk, as one of the risks in a model. If you have 10 years of good data (say), but the set includes an event that should only occur every 100 years. This level of impact is thus drastically overrepresented in the data and any fitted distribution will be biased toward such extremes. Yet the data point can not be completely ignored as such values can occur and the simulation models must have the capacity to sample such values (though with a reasonable likelihood). In this case the artistry that is fitting distributions to data comes to the fore. The data point could be removed from the set but not from our decision making process.

From the range of distributions that can be selected, the optimal choice should not only represent the remaining data well but also have a tail that samples events in the vicinity of those that have been excluded from the analysis with reasonable probability. No, that’s not always easy to do. But as with many elements of probabilistic modelling it simply must be done in order to provide useful information to decision makers.

Thus the context of the modelling can go a long way to determine the most appropriate steps to take with your data set. If that sounds like a subjective guideline then you read it correctly. Not enough people realise just how important experience and intuition can be in the seemingly prescriptive fields of mathematics and statistics. Fitting distributions to data is no different.

And yet that isn’t the unusual fact I was reminded of in the workshop! But I’ll leave that for Part 2 of my Data Issues blog.

Rishi Prabhakar
Trainer/Consultant

Digital Eyes on Alien Life

Wednesday, December 9, 2009 by Holly Bailey

University of Chicago geoscientist Patrick McGuire has big plans for Mars. Previously he worked on an imager for a Mars orbiter that could identify different types of soil and rock by detecting infrared and other wavelengths, and now he is drawing on that experience to develop a space suit with digital "eyes" and a neural network that rides on the hips of the spacesuit and can sort out living biological material from other matter.

The digital eyes will detect and plot colors, and the neural net, which is known as a Hopfield neural network, will compare these color patterns to a database of information previously gathered from that area of planet in order to make an animal-vegetable-mineral determination.

This complex AI system has already been tested at the Mars Desert Research Station in Utah, and McGuire and his colleagues were satisfied that the Hopfield algorithm could learn colors from just a few images and could recognize units that had been observed earlier.

McGuire's concept is that a human wearing this neural network could simply walk around the red planet and record every nearby object, rapidly gathering information.

Obviously, such a clothing item awaits a manned Mars mission. But in the meantime, why not have the next Rover suit up?

The Cat is Out of the Bag

Thursday, December 3, 2009 by Holly Bailey

At November's supercomputing conference in Portland, Oregon, IBM announced that its researchers working with a team from Stanford University had succeeded in developing an accurate simulation of human brain function. The simulation will be capable of emulating sensation, perception, action, interaction and cognition.

This algorithm simulating a living neural network, called BlueMatter (spelled as one word like everything else in computerese these days) is an important milestone in IBM's mission to build a cognitive computing chip because it begins to advance large-scale simulation of a cortical neural network and it synthesizes neurological data. BlueMatter is built with Blue Gene (two words for this pun in the singular) architecture, which, in combination with specialized MRI images, allowed the team to create a wiring diagram of the human brain. This map of the brain is, according to IBM's press release "crucial to untangling its vast communication network and understanding how it represents and processes information."

To be more accurate, what BlueMatter has thus far demonstrated is the potential to achieve neural network technology that operates on the scale of complexity of the human brain. The algorithm's current simulation approximates the cortical system of a cat. Hence, the title of the paper announcing IBM's accomplishments: "The Cat Is Out of the Bag." Even so, this is an operations research accomplishment that dwarfs such mundane analytical tasks as option valuation, value-at-risk, or reserve estimation.

One of the goals of the company's cognitive computing program is to create a chip that operates with the energy efficiency of the human brain (20 watts). But in order to emulate the brain activity of a cat, the research team had to bring out one of the largest supercomputers in the world, the IBM Dawn Blue Gene/P--which comprises about 150 thousand processors and contains 144 terrabytes of main memory.

This cat came out of a pretty big bag.

25 Worst Tech Products

Monday, November 30, 2009 by Steve Hunt

A friend and colleague who knows I write a Six Sigma blog sent me a link to an older article on PC World, The 25 Worst Tech Products of All Time that he thought might applicable to Six Sigma.

As first blush, I thought, “What an article on PCWorld.com on the Worst Tech products would have anything to do with Six Sigma?” The answer . . . everything! Particularly after reading the piece, the number 1 or worst product of all time (in their eyes) is American Online. I agree AOL has had its difficulties, but one has to admit the service has had staying power despite this. It’s been around for 20 years, which is a lifetime in the computer world. I don’t know if they utilized Voice of the Customer (VOC) , but they did something right since they are still around.

The article mentions AOL had shown improvements over the previous years. This goes to show us, they had a good idea, but took many years to sort out the bugs and for them to position themselves correctly. At the time of initial development they probably didn’t utilize Design for Six Sigma or another Critical Parameter development methodology, but it appears they may have implemented Lean Six Sigma principles to improve their “inexcusably poor customer service,” “inaccessible dial-up numbers,” and what I’ll call “flawed billing practices.” Please know I am not necessarily agreeing with the article, or being an advocate for AOL, I’m simply pointing out how the company has appeared to have improved its product and service over time.

One can only hope and assume that companies are doing a better job up front vetting their ideas, products and designs . . . with sound initiatives such as Design for Six Sigma. If not, hopefully we won’t seem the on PC World’s next “worst of” list.

If you would like to learn more about Design for Six Sigma, May I recommend either of these two free webinars:

Wayne Winston’s Math and Sports blog debuts on HuffPost

Thursday, November 12, 2009 by DMUU Training Team

Wayne Winston is the newest blogging personality at the Huffington Post! His first post, “The Importance of Schedule Strength in Sports,” appeared yesterday. Wayne will focus on the interface between math and sports, with detailed explanations of statistical analysis and spreadsheet modeling, including @RISK risk analysis models. You can find a link to the Wayne Winston blog from the newly-launched HuffPost Sports.

Wayne is the John and Esther Reese Professor of Decision Sciences at Indiana University’s nationally ranked Kelly School of Business. He has won over 30 teaching awards, and written over 20 journal articles and 15 books. Wayne has consulted for many organizations including the Dallas Mavericks, USA Diving, Cisco, Microsoft, US Army, Eli Lilly, Diamond Consulting, Tellabs and Medtronics. He has also developed online spreadsheet modeling and mathematics courses for Harvard Business School Publishing. And, Wayne is a two time Jeopardy! champion!

Wayne’s latest book, Mathletics, provides an introduction to the use of math by baseball, football, and basketball teams. He has also authored several books published by Palisade, including Financial Models Using Simulation and Optimization I, Financial Models Using Simulation and Optimization II: Investment Valuation, Options Pricing, Real Options & Product Pricing Models, and Decision Making Under Uncertainty with RISKOptimizer.

DMUU Training Team