Ready For What?

by
Shaun Doheney, Chair of Resources and Readiness Applications
Connor McLemore, Chair of National Security Applications

WantNail.png

For want of a nail the shoe was lost.
For want of a shoe the horse was lost.
For want of a horse the rider was lost.
For want of a rider the message was lost.
For want of a message the battle was lost.
For want of a battle the kingdom was lost.
And all for the want of a horseshoe nail.

The proverb “For Want of a Nail” describes how seemingly inconsequential details can lead to a disaster in military readiness, and is a valuable lesson for us all.  For those of us who make decisions or support decision-making involving risks or uncertainty, we need to have an answer to the question, “are we ready?”  Of course, that question should almost always be followed by the question, “ready for what?”  Are we ready to respond to the next natural disaster?  Are we ready to mitigate market volatility?  Is our energy infrastructure ready to handle the increased demand this summer?  Is our city ready for the expected increased growth over the next five years?

We (Connor McLemore and Shaun Doheney) have had military Operations Research experience, and have been working with Dr. Sam Savage here at ProbabilityManagement.org on an improved representation of military readiness. This provides a framework that we believe is useful, logically consistent, and most importantly is simple enough for adoption by military decision makers and those support such decision-making. As a poster child of poor military planning see the PowerPoint and Excel model describing the failed mission to rescue the American hostages in Iran in 1980.

OperationEagleClaw.png
 

One of the key components to this readiness representation framework is the ability to roll up readiness in a logical, mathematically sound, and intuitive way.  To paraphrase Dr. Savage in his recent blog titled, Why Was RiskRollup.com Available?, if squadron A has a 60% chance of accomplishing the mission and squadron B has a 70% chance, then if we send them both is there a 130% chance of success?

Recent improvements in our ability to account for uncertainty allow us to rethink approaches to representing military readiness.  To demonstrate our approach, we’ve created a few prototype models that you may download here

RolledUpReadiness.png
 

We hope that you’ll join us during the upcoming Military Operations Research Society (MORS) Symposium when we give presentations and a tutorial on this work.  While improved readiness accounting across the military and business or enterprises will likely be an evolutionary process with inputs from numerous stakeholders, the key in almost all situations is to “start small and reinforce success,” as Shaun likes to say.  And as Connor likes to say, “Go Navy; beat Army!”  But that’s a blog for another time!

© 2019 ProbabilityManagement.org

Datasaurus Arithmetic

Thank you, Alberto Cairo and Robert Grant

by Sam Savage

Datasaurus Arithmetic

Datasaurus Arithmetic

 
Data set HAP and PY

Data set HAP and PY

The three great milestones of manned flight were the Wright Brothers in 1903, the lunar landing in 1969, and the lithium ion laptop battery of the 1990s. This last breakthrough allowed me (while buckled into an airline seat to control my ADD) to develop a data set to dent the steam-era concept of correlation. I was on a flight from the East Coast to San Francisco, and over Denver I reached my goal: two variables, called HAP and PY, which had zero correlation, but nonetheless displayed a clear interdependency, as shown.

As I mentioned in my earlier blog on Virtual SIPs, I am not the only one poking fun at statistical concepts with ridiculous scatter plots. Alberto Cairo, a professor of Visual Journalism at the University of Miami, has a downloadable data set called Datasaurus, which has several X,Y pairs of data points, with identical summary statistics and correlation, but wildly different scatter plots. Alberto created his masterpieces with an interactive tool called DrawMyData from data scientist Robert Grant.

Never one to leave the bizarre well enough alone, I could not resist creating a model called Datasaurus Arithmetic, in which you may perform SIPmath calculations on the various patterns in Alberto’s dataset. Above we see the marginal distribution of X and Y (which I call Dino and saur), along with calculations involving the sum, product and quotient of X and Y while preserving the Jurassic joint distribution of X and Y.

If you teach statistics or data science, I urge you to download the file and compare the scatter plots and summary statistics of Alberto’s other included data sets.

Ⓒ 2019 Sam Savage

Hubbard/KPMG Enterprise Risk Management Survey

by Sam L. Savage

Hubbard Decision Research and KPMG have launched a short Risk Management survey, which I urge you to take and to forward to others before March 10. It only takes 6 – 7 minutes to fill out and will help us better understand this important but poorly defined field.

Doug will be presenting on The Failure of Risk Management at our Annual Conference in San Jose in March, and I am eager to get his first impression of the responses. And don’t forget that Tom Keelin, inventor of the Metalog distributions, will also be there. The next generation SIPmath Standard, which leverages Doug’s HDR Distributed Random Number Framework and Tom’s Metalogs, will facilitate a more quantitative approach to Enterprise Risk Management.

© Sam Savage 2019

Why Was RiskRollup.com Available?

RiskRollUp.png

by Dr. Sam Savage

Risk Doesn’t Add Up

If the risk of a power outage in City A next year is 60% and the risk of an outage in City B is 70%, then the risk of an outage across both cities is 130%, right? Obviously not, but what is it? Before the discipline of probability management, you couldn’t just add up risks. But today, you can represent the uncertainty of an outage in each city as a SIP as shown, where a 1 indicates an outage in that city. Simply summing the SIPs row by row provides the number of failures across both cities, then using the “Chance of Whatever” button in the SIPmath Tools you will find that that the risk of at least one failure across both cities is 88%. This pastes the following formula into the spreadsheet.

=COUNTIF( Sum, ">=1") / PM_Trials, where PM_Trials is the number of trials.

I am currently working with Shaun Doheney and Connor McLemore to apply these idea to Military Readiness, and Shaun will be presenting the MAP Model at our upcoming Annual Conference.

Nobody Has a Clue That This is Possible

How do I know? I recently bought RiskRollup.com, ConsolidatedRiskManagement.com, and ConsolidatedRiskStatement.com for $11.99 each. I probably won’t be able to retire on these investments, but I’ll bet I get a decent return.

Probability Management is Stochastic Optimization Without the Optimization

The holy grail of consolidated risk management is to optimize a portfolio of mitigations to provide the best risk reduction per buck. You might think that if people aren’t even rolling up risk today, we must be years away from optimizing. But that is not true. The concept of SIPs and SLURPs was in use in the field of stochastic optimization (optimizing under uncertainty) long before probability management was a gleam in my eye. This is the technique we applied at Royal Dutch Shell in the application that put probability management on the map. The scenarios of uncertainty generated by stochastic optimization are effectively SLURPs, and I argue that they are too valuable in other contexts not to be shared in a corporate database.

We are honored that a pioneer in stochastic optimization, Professor Stan Uryasev of the University of Florida, will also be presenting at our Annual Conference.  I know I have a lot to learn from him. I hope you will join us in March.

More on rolling up risk and a discussion of the Consolidated Risk Statement are contained in a December 2016 article in OR/MS Today.

Ⓒ 2019 Sam Savage

Virtual SIPs

The Generator Generator

by Sam L. Savage

GeneratorGenerator.png
 

Distribution Distribution

Decades ago, I discovered that few managers were benefiting from probabilistic analysis. Despite widely available simulation software such as @RISK and Crystal Ball, most people lacked the statistical training required to generate the appropriate distributions of inputs. 

“But wait a minute,” I thought to myself. “The general public still uses light bulbs even though they don’t know how to generate the appropriate electrical current.” After some research I discovered that there is a power distribution network that carries current from those who know how to generate it to those who just want to use it.

So why not create a Distribution Distribution network, to carry probability distributions from the people who know how to generate them (statisticians, econometricians, engineers, etc.) to anyone facing uncertainty?

Great idea, but it took me a while to figure out the best way to distribute distributions.  Eventually I arrived at the SIPs and SLURPs of probability management, which represent distributions as vectors of realizations and metadata which support addition, multiplication, and any other algebraic calculation, while capturing any possible statistical relationship between variables. This concept even works with the data set invented by Alberto Cairo, made up of SIPs I call Dino and saur [i].

A Scatter Plot of Alberto Cairo’s Dino and saur

A Scatter Plot of Alberto Cairo’s Dino and saur

 

Once Excel fixed the Data Table, it became possible to process SIPs in the native spreadsheet, which greatly accelerated adoption [ii]. SIPs and SLURPs have been a simple, robust solution, although they do require a good deal of storage.

Before I thought of SIPs, I had thought of and abandoned an idea involving snippets of code which would generate a random number generator when they arrived on a client computer.  I called this approach the Generator Generator (well, that was for short—the full name was the Distribution Distribution Generator Generator). The advantage of such a system is that the storage requirements would be tiny compared to SIPs, and you could run as many trials as you liked. It might not be possible to capture the interrelationships of Dino and saur, but at least some forms of correlations could be preserved.

The SIPmath/Metalog/HDR Integration

Recent breakthroughs from two comrades-in-arms in the War on Averages have made the Generator Generator a reality and allowed it to be incorporated into the SIPMath Standard. One key ingredient is Tom Keelin’s amazingly general Metalog System for analytically modeling virtually any continuous probability distribution with one formula.

Another is Doug Hubbard’s latest Random Number Management Framework, which in effect can dole out independent uniform random numbers like IP addresses while maintaining the auditability required by probability management. This guarantees that when global variables such as GDP are simulated in different divisions of an organization, they will use same random number seed. On the other hand, when simulating local variables, such as the uncertain cost per foot of several different paving projects, different seeds will be guaranteed. This allows individual simulations to be later aggregated to roll up enterprise risk. Doug’s latest generator has been tested thoroughly using the rigorous dieharder tests [iii].

At ProbabilityManagement.org, we have wrapped these two advances into the Open SIPmath Standard for creating libraries of virtual SIPs, which will take up a tiny fraction of the storage of current SIP libraries. We hope to release the tools to create such libraries at our Annual Meeting in San Jose on March 26 and 27. Tom, Doug, and I will be presenting there, along with an all-star cast of other speakers. I hope we see you there.

© Copyright 2019, Sam L. Savage

All-Star Lineup for our 2019 Annual Conference

loring_ward_3.jpg

by Sam Savage

Applications of Probability Management
March 26 - 27, 2019
San Jose, CA

SIPmath is a broad-spectrum cure for the Flaw of Averages, which impacts all plans involving uncertainty. With this in mind, our 2019 Annual Conference casts a wide net over a variety of probability management applications. I urge you to look through the abstracts.

 We have many great speakers lined up, including:

  • Deborah Gordon – Director, City/County Association of Governments, San Mateo County

  • Max Henrion – CEO of Lumina Decision Systems and 2018 Ramsey Decision Analysis Medal Recipient

  • Doug Hubbard – author of How to Measure Anything and The Failure of Risk Management

  • Tom Keelin – Inventor of the Metalog Distribution & Chief Research Scientist at ProbabilityManagement.org

  • Michael Lepech – Associate Professor of Civil and Environmental Engineering, Stanford University

  • Harry Markowitz – Nobel Laureate in Economics (via live webcast)

  • Greg Parnell – Military Operations Researcher & Professor at the University of Arkansas

  • Stan Uryasev – Risk Management Expert & Professor at the University of Florida

Topics covered include:

  • Analytics Wiki Development

  • Applying in SIPmath in Human Relations

  • Military Readiness

  • Municipal Risk Management

  • Applied Economics

  • Probabilistic Energy Forecast

  • Bridge Safety

  • Water Management

Register by Friday, February 1 to take advantage of our early registration discount.

Video Excerpts: Probability Management at Stanford University

SCPD Logo.png
 

by Sam Savage

On September 17, I delivered a one-hour webinar previewing my Winter Quarter course in Project Risk Analysis in Stanford University’s Department of Civil and Environmental Engineering. This course will apply the discipline of probability management to such problems as risk return tradeoffs in R&D portfolios and rolling up operational risk across assets such as gas pipelines. Although the entire 57-minute webinar is available, I recommend the following excerpts.

 

The "Chance of Whatever" Button

Defense against “Give me a Number”

by Sam Savage

ChanceofWhateverArrow.png

A common fork in the road to hell is arrived at when, in the face of uncertainty, the boss demands: “Give me a number.” You may be tempted to respond with, “Would you settle for an average?” But even the correct average of the uncertain duration of a task, demand for a new product, or labor hour requirements for a job, leads to a host of systematic errors that guarantee that your plans will be wrong on average. I dubbed this problem “The Flaw of Averages” in an article in the San Jose Mercury News in 2000, and have been struggling to correct it ever since with growing success.

Technically you should say to the boss, “Here’s the probability distribution of the number you want.” But I don’t recommend that if you want to keep your job. Instead, the latest version of the SIPmath™ Modeler Tools, both the free version and guilt-free $500 Enterprise version, now include the new “Chance of Whatever” button.

Just put your cursor in the cell where you want the chance of whatever to appear, then specify the uncertain cell that needs to be greater or less than your boss’s specified goal. Then click OK. Now as you change your goal, the chance cell will immediately update. So, next time the boss demands a number, you can respond with, “What do you want it to be? I can tell you the chance of meeting your goal.”

Brian Putt, Chair of Energy Practice at ProbabilityManagement.org, has a new video on how to use this feature of our tools. Check it out.

 

 
© Copyright 2018 Sam Savage

Tom Keelin Named Chief Research Scientist

by Sam Savage

Tom Keelin

Tom Keelin

We are happy to announce that Tom Keelin, inventor of the Metalog system, will join ProbabilityManagement.org as Chief Research Scientist. Tom is Founder and Managing Partner at Keelin Reeds Partners, former Worldwide Managing Director of Strategic Decisions Group, and co-founder of Decision Education Foundation. He holds a PhD in Engineering-Economic Systems from Stanford University.

On their own, Metalogs represent an unprecedented, unified approach to creating analytical formulas to represent probability distributions derived from data. Coupled to the HDR Random Number Management Framework from Doug Hubbard, they are leading to a new generation of SIPmath in which SIP libraries, which currently may contain millions of data elements, will be reduced to a few lines of code. These in turn will create virtual SIPs on an as-needed basis, without losing the fundamental properties of additivity and auditability that are the hallmarks of the discipline of probability management.

Watch for an upcoming blog post on the combined use of the SIPmath, HDR, and Metalog standards.

Related Reading: Tom Keelin’s Metalog Distributions

© Copyright 2018 Sam Savage

None of My Successes Have Been Planned and None of My Plans Have Been Successful

Simulating Rags to Riches and Vice Versa

by Sam Savage

Blog_HeatMap.png
 

Planning vs. Scheming

Since much of my income is from consulting, I have devoted resources to reaching out to appropriate clients. I can’t count the number of engagements I’ve gotten this way because there aren’t any. All my engagements have dropped in from out of the blue.

“But how about your 2009 book?” you say. “That was marketing on a grand scale. Some would have even called it selling out. You must have had customers breaking down your door after that.”

Nope. There was a horrific worldwide recession and I lost my key clients instead of getting new ones.

“But things are going great now, right?” Absolutely, and I am deeply thankful. But this was due to dumb luck, such as the improved Data Table function in Microsoft Excel, which enabled SIPmath, and stumbling upon adult supervision in the nick of time.

None of my successes have been planned and none of my plans have been successful. So, I don’t plan (much to the consternation of my adult supervisors). Instead, I scheme, by putting options in place in case the appropriate planets align. However, Louis Pasteur said that “Chance favors the prepared mind,” and I do try to prepare my mind. I just don’t plan.

So, when I heard that three Italian physicists (Pluchino, Biondo, & Rapisarda) had written a paper called “Talent vs Luck: the role of randomness in success and failure,” I was all ears. Among other things, they address the question of why, if talent is distributed along a bell curve, that wealth is extremely skewed with the top few percent of the population owning the lion’s share. They created a simulation that shows how chance drives the disparity in the distributions of talent and wealth. Inspired by the physicists, Dave Empey [1] and I built our own SIPmath model in Excel (available on our Models page) to explore similar principles. Our model shows that chance plays a role, but that disparity in income can arise without it. NOTE that unlike the physicists’ model, ours is not calibrated to reality, and is merely designed to give directional results.  

The Model

Free models, like free advice, are worth what you pay for them. The admonition of George Box, that “all models are wrong, but some are useful,” applies in spades to economics, where Chaos Theory is always lurking a few decimal places away. I think the Italians would agree with me that such models do not provide “right answers” as much as “right questions.”

 

With the above caveats in mind, our model has the following elements.

1. We start with 50 agents, whose talents are measured in IQ score, normally distributed with mean of 100 and standard deviation of 15. These are assigned at the beginning and do not change during the simulation.

Blog_Talent.png
Blog_Wealth.png
 

We also endow the agents with an initial wealth distribution, which may be uniform, or skewed either toward the high or low intelligence agents.

Blog_Additive.png

2. . We then simulate two forms of IQ-based income (wealth accumulation) over twenty years; either adding wealth proportional to IQ or multiplying wealth by a factor proportional to IQ. In either case the user may specify a degree of uncertainty from year to year.

3. We also allow for additional Chance Events that can impose independent positive or negative impacts for each agent.

Blog_HeatMap2.png

4. A heatmap displays the relative wealth by year each agent for a single trial. It is fun to crank up the uncertainty, press the <calculate> key, and watch the unsuspecting agents succeed or fail beyond their wildest simulated dreams.

5. Given the above calculations, we run 100 simulated trials of final wealth for each of the 50 agents, effectively generating a simulated population of 5,000 agents over which we calculate the final wealth distribution.

Results

A key result of Pluchino, Biondo, & Rapisarda is that the final wealth in their simulation (which was more complex than ours) was very skewed even though talent was normally distributed.  Our model indicates that you can’t sneeze without creating a skewed distribution of final wealth. For example, suppose there is no uncertainty, and all agents start with equal wealth, that increases each by a percentage proportional to their IQ. This is analogous to agents with investments that grow at different rates. Then you get the distribution of final wealth shown below.  

Blog_FinalWealth.png
Blog_PercentofWealth.png

Here we have the top 1% of the population holding 10% of the wealth. Adding additional uncertainties makes the skew worse, but talk is cheap, I suggest that you download the model here and play with it yourself.


[1] Director of Software Development at ProbabilityManagement.org and programmer of the SIPmath™ Modeler Tools.

© Copyright 2018 Sam Savage