You’ve Got a “Noise” Problem and You Don’t Even Know It

getAbstract International Book Award 2021: Turn down the volume with some good old-fashioned methodical thinking.

You’ve Got a “Noise” Problem and You Don’t Even Know It

Humans fall prey to errors in judgment all the time. Often these relate to cognitive biases, but they can also be due to “noise.” Psychologist and Nobel Prize-winner Daniel Kahneman, along with his Noise co-writers Olivier Sibony and Cass R. Sunstein, tease out a problem so common to human interaction it’s been hiding in plain sight. That problem is “noise.” 

Humans Are “Noisy” Thinkers

When experts come to different judgments that should be identical, that’s “noise,” and it costs businesses plenty. Insurance executives acknowledged they would expect to see some variation in the premiums their adjusters assigned similar accounts. After all, they’re human. They estimated amounts would vary by about 10%. In a test where adjusters evaluated the same five cases, their judgments varied instead by 55%, a shockingly large difference. In this example, the company is potentially leaving money on the table due to the “noisiness” of its decision-making process.

What people usually claim to strive for in verifiable judgments is a prediction that matches the outcome.

Daniel Kahneman, Olivier Sibony and Cass R. Sunstein

For defendants awaiting judicial sentencing, the stakes of this variability are much higher. Two men who cashed counterfeit checks in close to the same amounts, each without a prior criminal record, drew two different judges. One was sentenced to 15 years in jail, the other to 30 days. The “noise” within the legal system leads to variations in judgments that cannot all be just.

Image of: Noise
Book Summary

Noise

Wherever judgment exists, you will also find noise – and more of it than you think.

Daniel Kahneman, Olivier Sibony and Cass R. Sunstein Little, Brown Spark
Read Summary

Analyzing research results across fields, Kahneman, Sibony and Sunstein found examples of noisy judgment in medical diagnoses, forecasting of all kinds, forensic science and hiring, to name a few. Their conclusion is noise in decision-making is epidemic and has far-reaching consequences.

Bias Versus Noise

In his best-selling book Thinking Fast and Slow, Kahneman mapped out for readers all the ways in which their thinking is vulnerable to bias. The problem is people mostly use “System 1” thinking – which is faster but more error-prone – rather than the more conscious and meticulous “System 2” thinking. Because System 1 is based on experience, emotions and intuition, people tend to trust it more. Ultimately, it’s more efficient. However, if you want to be sure of your conclusions, you need “System 2,” slow, thinking. This is the system scientists purposefully use to formulate and test hypotheses.

Image of: Thinking, Fast and Slow
Book Summary

Thinking, Fast and Slow

“Two systems” vie for control of your mind, and “two selves” decide your happiness. Can all four of you ever get along?

Daniel Kahneman Farrar, Straus & Giroux
Read Summary

In the absence of conscious reassessment, it’s easy for biases to creep into human judgment. It’s a common bias, for instance, to start with a conclusion then find the evidence to support it. Other common biases include replacing a difficult question with a simpler one and refusing to change quickly-formulated impressions. Consistent bias can lead to system bias but, because different people have different biases, taken together they add up to system-wide noise.

“Noise” includes biases but can be as subtle as the influence a football team’s win or loss has on a judge’s mood, a person’s particular experience or what day of the week it is. Imagine shooters and their targets. Biased shooters will miss the bull’s-eye, but all in the same pattern. Noisy shooters will produce a random pattern. “Noise” is hard to measure for but it can be controlled for.

Multiple, conflicting cues create the ambiguity that defines difficult judgment problems.

Daniel Kahneman, Olivier Sibony and Cass R. Sunstein

Fix Your Noise Problem

People make “predictive” and “evaluative” judgments. Both are prone to varying levels and categories of errors. To begin to correct for noise, the authors recommend organizations conduct a “noise audit.” Have several professionals diagnose the same problem. The variation in their conclusions indicates “noise.” Armed with some data about where your company makes its noisiest decisions, you can create a blueprint for “decision hygiene.” Audits can measure human versus machine judgments to determine accuracy.

Artificial intelligence (AI) can crunch through enormous amounts of data to make predictions that are more accurate than human prediction, although people are reluctant to rely upon machine judgment. However, human judgment often comes down to a professional guess, a hunch. Sometimes the hardest thing for people to do is admit their ignorance, but it’s the first step to perceiving noise and finding solutions.

Bias leads to errors and unfairness. Noise does too – and yet, we do a lot less about it.

Daniel Kahneman, Olivier Sibony and Cass R. Sunstein

Prevent “noise” with good “decision hygiene”:

  • By carefully sequencing how you evaluate information, looking at only what you need when you need it, you can put off forming intuitions.
  • When making predictions, utilize the work of several predictors to find a closer-to-accurate average.
  • Develop guidelines, standards and checklists to help you diagnose and work through problems.
  • Break problems or decisions into component pieces, which focuses evaluation.

For hiring, a process rife with biases, you can pre-define the questions you ask prospective employees. Delay including your own intuition in the process as long as possible. Incorporating Kahneman’s Systems thinking insights in “A Structured Approach to Strategic Decisions,” his co-author Olivier Sibony, along with Dan Lovallo, outlines the Mediating Assessments Protocol (MAP) to help organizations methodically think through decision-making so as to minimize bias and errors.

Image of: A Structured Approach to Strategic Decisions
Article Summary

A Structured Approach to Strategic Decisions

Learn a Nobel Prize winner’s system for making more logical, objective and reliable strategic decisions.

Daniel Kahneman, Dan Lovallo and Olivier Sibony MIT Sloan Management Review
Read Summary

Sometimes “noise” is good. While algorithms make fewer errors than human judgments, they don’t always reflect human values. AI solutions may be cost-prohibitive. Sometimes you want several teams working to solve the same problem because you want the variation in conclusions, it may lead to better innovation. Incorporating Kahneman’s “Systems” thinking, co-writer Cass R. Sunstein joined Eric Singler and Pierre Chandon in writing Nudge Management, which illustrates how managers can think through critical decisions methodically, but implement them in ways that work with instead of against human nature.

Image of: Nudge Management
Book Summary

Nudge Management

Consultant Eric Singler covers employee engagement and performance with top research and references.

Eric Singler, Cass R. Sunstein and Pierre Chandon Pearson France
Read Summary

Find more great summaries to help you make better decisions:

Image of: Decision Making
Channel

Decision Making

How to face tough choices with strategy and sense.

Open Channel
Share this Story