You’ve Got a “Noise” Problem and You Don’t Even Know It
Humans fall prey to errors in judgment all the time. Often these relate to cognitive biases, but they can also be due to “noise.” Psychologist and Nobel Prize-winner Daniel Kahneman, along with his Noise co-writers Olivier Sibony and Cass R. Sunstein, tease out a problem so common to human interaction it’s been hiding in plain sight. That problem is “noise.”
Humans Are “Noisy” Thinkers
When experts come to different judgments that should be identical, that’s “noise,” and it costs businesses plenty. Insurance executives acknowledged they would expect to see some variation in the premiums their adjusters assigned similar accounts. After all, they’re human. They estimated amounts would vary by about 10%. In a test where adjusters evaluated the same five cases, their judgments varied instead by 55%, a shockingly large difference. In this example, the company is potentially leaving money on the table due to the “noisiness” of its decision-making process.
What people usually claim to strive for in verifiable judgments is a prediction that matches the outcome.
Daniel Kahneman, Olivier Sibony and Cass R. Sunstein
For defendants awaiting judicial sentencing, the stakes of this variability are much higher. Two men who cashed counterfeit checks in close to the same amounts, each without a prior criminal record, drew two different judges. One was sentenced to 15 years in jail, the other to 30 days. The “noise” within the legal system leads to variations in judgments that cannot all be just.
Analyzing research results across fields, Kahneman, Sibony and Sunstein found examples of noisy judgment in medical diagnoses, forecasting of all kinds, forensic science and hiring, to name a few. Their conclusion is noise in decision-making is epidemic and has far-reaching consequences.
Bias Versus Noise
In his best-selling book Thinking Fast and Slow, Kahneman mapped out for readers all the ways in which their thinking is vulnerable to bias. The problem is people mostly use “System 1” thinking – which is faster but more error-prone – rather than the more conscious and meticulous “System 2” thinking. Because System 1 is based on experience, emotions and intuition, people tend to trust it more. Ultimately, it’s more efficient. However, if you want to be sure of your conclusions, you need “System 2,” slow, thinking. This is the system scientists purposefully use to formulate and test hypotheses.
In the absence of conscious reassessment, it’s easy for biases to creep into human judgment. It’s a common bias, for instance, to start with a conclusion then find the evidence to support it. Other common biases include replacing a difficult question with a simpler one and refusing to change quickly-formulated impressions. Consistent bias can lead to system bias but, because different people have different biases, taken together they add up to system-wide noise.
“Noise” includes biases but can be as subtle as the influence a football team’s win or loss has on a judge’s mood, a person’s particular experience or what day of the week it is. Imagine shooters and their targets. Biased shooters will miss the bull’s-eye, but all in the same pattern. Noisy shooters will produce a random pattern. “Noise” is hard to measure for but it can be controlled for.
Multiple, conflicting cues create the ambiguity that defines difficult judgment problems.
Daniel Kahneman, Olivier Sibony and Cass R. Sunstein
Fix Your Noise Problem
People make “predictive” and “evaluative” judgments. Both are prone to varying levels and categories of errors. To begin to correct for noise, the authors recommend organizations conduct a “noise audit.” Have several professionals diagnose the same problem. The variation in their conclusions indicates “noise.” Armed with some data about where your company makes its noisiest decisions, you can create a blueprint for “decision hygiene.” Audits can measure human versus machine judgments to determine accuracy.
Artificial intelligence (AI) can crunch through enormous amounts of data to make predictions that are more accurate than human prediction, although people are reluctant to rely upon machine judgment. However, human judgment often comes down to a professional guess, a hunch. Sometimes the hardest thing for people to do is admit their ignorance, but it’s the first step to perceiving noise and finding solutions.
Bias leads to errors and unfairness. Noise does too – and yet, we do a lot less about it.
Daniel Kahneman, Olivier Sibony and Cass R. Sunstein
Prevent “noise” with good “decision hygiene”:
- By carefully sequencing how you evaluate information, looking at only what you need when you need it, you can put off forming intuitions.
- When making predictions, utilize the work of several predictors to find a closer-to-accurate average.
- Develop guidelines, standards and checklists to help you diagnose and work through problems.
- Break problems or decisions into component pieces, which focuses evaluation.
For hiring, a process rife with biases, you can pre-define the questions you ask prospective employees. Delay including your own intuition in the process as long as possible. Incorporating Kahneman’s Systems thinking insights in “A Structured Approach to Strategic Decisions,” his co-author Olivier Sibony, along with Dan Lovallo, outlines the Mediating Assessments Protocol (MAP) to help organizations methodically think through decision-making so as to minimize bias and errors.
Article Summary
A Structured Approach to Strategic Decisions
MIT Sloan Management ReviewSometimes “noise” is good. While algorithms make fewer errors than human judgments, they don’t always reflect human values. AI solutions may be cost-prohibitive. Sometimes you want several teams working to solve the same problem because you want the variation in conclusions, it may lead to better innovation. Incorporating Kahneman’s “Systems” thinking, co-writer Cass R. Sunstein joined Eric Singler and Pierre Chandon in writing Nudge Management, which illustrates how managers can think through critical decisions methodically, but implement them in ways that work with instead of against human nature.
Find more great summaries to help you make better decisions:
How the Journal Saves You Time
Reading Time
5
min.
Reading time for this article is about 5 minutes.
Saved Time
26
h
This article saves you up to 26 hours of research and reading time.
Researched Abstracts
4
We have curated the most actionable insights from 4 summaries for this feature.
1
1 Article
3
We read and summarized 3 books with 1266 pages for this article.
