Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 47 ยท Heuristics and Biases

Overconfidence and confirmation bias

Learn the three kinds of overconfidence, calibration, the planning fallacy, the Wason 2-4-6 task and confirmation bias, with a calibration quiz and simulations.

Two habits of mind quietly shape almost every judgment. We tend to be more sure of our answers than their accuracy warrants (overconfidence), and we tend to look for evidence that supports what we already believe (confirmation bias). Together they help explain why forecasts miss, projects run late, and disagreements harden.

In this lesson you will see how researchers measure confidence, why the same bias takes different forms, and how experts in a few fields avoid it. You will also test your own calibration with a short quiz.

Calibration chart: stated confidence runs ahead of accuracy
Caption: a schematic calibration curve. Perfect calibration lies on the dashed diagonal; typical overconfidence lies below it.

What you will learn

  • The three forms of overconfidence: overestimation, overplacement and overprecision.
  • What calibration is and how it is measured.
  • How the planning fallacy and the Wason 2-4-6 task illustrate overconfidence and confirmation bias.
  • Techniques that help: feedback, pre-mortems and the outside view.

Key terms

Calibration: the match between how confident you are and how often you are right.

Overprecision: being too sure that your estimate is accurate, for example setting intervals that are too narrow.

Overplacement: believing you are better than others when you are not.

Planning fallacy: underestimating the time, cost and risk of your own future projects.

Confirmation bias: seeking, interpreting and remembering evidence in ways that support existing beliefs.

Three kinds of overconfidence

Don Moore and Paul Healy (2008) separated three things that are often lumped together.

  1. Overestimation: thinking your performance is better than it is (you expect a score of 80 and get 65).
  2. Overplacement: thinking you rank higher than others. In a well-known study by Ola Svenson (1981), a large majority, 93% of American participants, rated their driving skill as above the median of the group, which cannot be true for everyone.
  3. Overprecision: being too sure of the exact truth. This is the most consistent form: people give intervals that miss the truth far more often than they claim.

Interestingly, overestimation tends to be strongest on hard tasks and overplacement on easy tasks. On hard tasks people underplace themselves, because they assume everyone finds it hard too. Overconfidence is not one trait but a family of effects.

Measuring calibration

Ask people for a 90% confidence interval ("I am 90% sure the answer lies between X and Y") for a series of quantities. If they are well calibrated, nine out of ten intervals contain the truth. In many studies, starting with Marc Alpert and Howard Raiffa's work in the 1960s, only about half the intervals contain the truth. Intervals are too narrow, so people are overprecise.

A second method groups probability judgments: among all the times you said "80% sure," how often were you right? Weather forecasters, who get quick and clear feedback each day, are among the best calibrated professionals: when they say 70% chance of rain it rains about 70% of the time. Feedback and practice improve calibration, and the effect shows up in the chart above.

The planning fallacy

Daniel Kahneman and Tversky named the planning fallacy: people predict completion times by imagining the best-case plan and neglect delays that happen to nearly everyone. In a study by Roger Buehler, Dale Griffin and Michael Ross (1994), students estimated how long it would take to finish their thesis; the average estimate was a bit over a month, while the actual average was almost two months. Large public projects show the same pattern. The remedy proposed by Kahneman and Dan Lovallo is the outside view (or reference class forecasting): instead of building up an estimate from your plan, look at how long similar projects actually took.

Confirmation bias

In 1960 Peter Wason gave people the triple 2, 4, 6 and told them it followed a rule. They could propose their own triples and would be told whether each fit the rule. Most guessed "numbers increasing by 2" and tested triples like 8, 10, 12 and 20, 22, 24. Every test said "yes," so they became confident. The actual rule was simply "any increasing sequence." They almost never tested a triple that would violate their hypothesis (like 1, 2, 3 or 6, 4, 2) and so never discovered the broader rule.

This is positive testing, and it is one face of confirmation bias. Others include interpreting ambiguous evidence in favor of your view and remembering supportive facts better. In 1979 Charles Lord, Lee Ross and Mark Lepper gave people who supported or opposed capital punishment the same mixed evidence; each side judged the evidence supporting its own view as stronger, and some became more polarized. Later work shows that polarization is not inevitable, but biased evaluation of evidence is common.

Confirmation bias is not always irrational. When searching for a hypothesis, testing where it is most likely to be true can be efficient. The trouble starts when we never test where it would fail.

What helps

  • Feedback with clear scoring. Keep a record of predictions with probabilities and score them later (a "forecasting journal").
  • Pre-mortem. Before a project starts, imagine it has failed and write why. Psychologist Gary Klein proposed this method to surface risks people do not voice.
  • Consider the opposite. Force yourself to write the best case against your view.
  • The outside view. Use data on similar past cases.
  • Wide intervals. Practice making intervals wide enough that you are surprised about 10% of the time.

Common misconceptions

  • "Overconfidence is arrogance." It affects humble people too, and it is often not a personality trait but a feature of how we judge evidence.
  • "Experts are immune." Experts can be overconfident, especially in areas without clear feedback. Weather forecasters, with daily feedback, are an exception.
  • "Confirmation bias means never testing your beliefs." It means testing them lopsidedly.

Try it

First, a calibration quiz. Give a range for each quantity that you are 90% sure contains the true value, then reveal the answers. This is a self-assessment demonstration, not a scientific test; with only five questions the result is noisy.

Calibration quiz (90% intervals)

Second, a simulation of the Wason 2-4-6 task. The hidden rule is "any increasing triple". A tester believes "increasing by 2". We compare a tester who only tries triples that fit the belief with one who also tries triples that break it.

Python Positive testing versus testing for failure (a simulation)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A person who gives 90% intervals that contain the true value only about half the time shows

    Intervals that are too narrow reveal overprecision, the most consistent form of overconfidence.
  2. 2. The planning fallacy is best reduced by

    Reference class forecasting corrects the neglect of delays that happen to similar projects.
  3. 3. In Wason's 2-4-6 task, why did most participants fail to discover the true rule?

    Positive testing always returns 'yes' and never exposes that the rule is broader.
  4. 4. Why are weather forecasters unusually well calibrated?

    Daily feedback and scoring let them learn to match confidence to accuracy.
  5. 5. A pre-mortem asks you to

    Imagining failure in advance surfaces risks that optimism tends to hide.

Summary

  • Overconfidence comes in three forms (overestimation, overplacement, overprecision); overprecision is the most consistent.
  • Calibration compares confidence with accuracy; feedback improves it.
  • The planning fallacy underestimates time and cost; use the outside view.
  • Confirmation bias shows up as positive testing and biased evaluation of evidence.
  • Helpful practices: forecasting journals, pre-mortems, considering the opposite and wide intervals.

Next: Chapter 3 review