In 1914, Professor John Alexander Smith addressed the first session of his two-year lecture course at Oxford as follows:
Gentlemen, you are now about to embark on a course of studies that (will) form a noble adventure…Let me make this clear to you…nothing that you will learn in the course of your studies will be of the slightest possible use to you in after life – save only this – that if you work hard and intelligently, you should be able to detect when a man is talking rot, and that, in my view, is the main, if not the sole purpose of education.
This brilliantly insightful statement underscores a growing challenge for both society and business. So what is the problem? Well, the problem is that fewer and fewer individuals are getting anything close to a good education – at least with respect to a sound understanding of statistics and measurement.
I see a concerning state of affairs within many businesses because of a startling lack of understanding of statistics and the psychometric notions of reliability and validity associated with measurement. The result has been a stunning amount of gullibility: if we put a chart or graph in front of many business leaders and start with the often meaningless phrase “research shows,” all crap detection goes out the window.
And here is the big reason this is such a problem: GIGO (garbage in; garbage out). When we accept flawed or incomplete input data as being valid, or account for only a few of the potential impacting variables, we make decisions on a cockamamie intellectual foundation. Predictably, consequences range from disappointing to devastating.
One can then add to that the challenge of sometimes having to deal with individuals who are neither naïve nor incompetent, but rather purposefully make use of selective data to advance a desired outcome. Such individuals present a major problem, because they are often the ones who most loudly and assertively try to impose their beliefs on others by “proving” their point with “data.” This pattern is easy to see in the current COVID situation. You can find charts (usually supplied by authors who use a lot of exclamation points and ALL CAPS) that suggest all is fine. You will also be shown charts that suggest we are at the precipice of utter disaster. It is likely the truth falls somewhere in between.
Many in the business world, often due to legal concerns, seem to be on a quest for objectivity, i.e., they want to base decisions on hard data. This makes a lot of sense when reliable and valid data are available, but alas, the world is not always amenable to our often meager and at best imprecise attempts at measurement.
In my consulting role at MindSet, a few areas where I see these decision-making errors most commonly manifest are:
- staff evaluations that try to reduce complex human interactions to simple forms and metrics
- employee selection processes that rely on questionable personality assessment instruments
- when considering ROI, ignoring factors that do not lend themselves to easy measurement and tabulation
- projected returns on investment that are predicated on overly optimistic assumptions and understated risk assessments
- exaggerated projections of savings or productivity gains from capital expenditures
- selective or too narrow measurement of customer satisfaction
- inadequate sampling, poorly written items, or poor process used with employee surveys
- staff turnover analysis that fails to properly weight employee value to the company
- aggregate exit interview data that fails to properly weight information based on the quality of the source
It is at those times that we would be wise to remind ourselves of Professor Smith’s admonition: being able to identify rot as rot is one of the more valuable skills a business leader can have.
A Probability Thought Experiment
Imagine a magical casino. At one of the tables there is a game called “Shuffle for Your Life”. The wager is this: With perfect randomization, the dealer will shuffle two decks of cards and lay them out next to each other. If they don’t match exactly, you win a quarter ($0.25). If they do match, you die. Is it a smart bet?
Actually, it probably is.
Even smarter would be for you to offer to raise the stakes by asking to put in for multiple plays while you hit the buffet. If that’s allowed, then offer to let them keep the game running for the rest of your life and to send you the proceeds weekly. In fact, you should go to every table in the casino and see if they will make the same bet with you. Find as many ways as possible to keep taking this bet. Tell your friends to do the same. Milk this magical casino for all it’s worth.
Then head home and sleep soundly knowing that the risk of the game killing you, while not exactly zero, is so close to zero that it will probably never happen even if the casino enlisted every single person on the planet to shuffle decks on their behalf for the rest of your life.
Explanation: Let’s assume a continuous, around the clock, shuffle per second. To get the perfect match between the two decks, it is projected to take about 1.28 x 1060 years or 9.13 x 1049 X 14,000,000,000 (14 billion years is roughly the age of the universe, give or take a decade) years to have a 50% chance it would happen.
Put more succinctly, the probability of a match is 1 in 52 factorial (52 x 51 x 50 x 49 x ….. x 2 x 1). For a fun visual explanation of just how inconceivably large of a number that is, view this 5-minute video.
So should that offer ever come along, take the quarter, sleep well, and enjoy your new-found wealth.
Founder of MindSet, LLC.
