Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 53 ยท Safety, Risks and Responsibility

Hallucinations and How to Reduce Them

Understand why language models make things up, then reduce hallucinations with grounding, verification, confidence signals and self-consistency checks.

A hallucination is output that sounds fluent and confident but is false, unsupported or invented: a book that does not exist, a legal case with a made-up citation, a function that is not in the library. Hallucinations are not rare glitches that a future version will simply remove. They follow from how language models work, so every application that depends on factual accuracy needs a plan for them.

This lesson explains why they happen, the main types, and the practical defenses: grounding, verification, confidence signals and self-consistency checks. It builds on How LLMs Are Trained and Logits, Softmax and Temperature.

Why models make things up

A language model is trained to predict a plausible next token, not to look facts up. Several forces push it toward invention:

  • Plausibility, not truth. A fluent sentence about a real-sounding author is a high-probability continuation, even if no such author exists.
  • Gaps and rare facts. Facts that appear seldom in training data are stored weakly. The model fills the gap with something that fits the pattern.
  • No built-in "I don't know". Training rewards answering. Unless the model was specifically tuned to abstain, a missing fact still produces text.
  • Knowledge cutoff. The model knows nothing after its training data ends, but may answer as if it did.
  • Sampling. Each token drawn at random from the distribution can steer the text toward a wrong branch, and later tokens then build on the mistake.
  • Pressure from the prompt. A question with a false premise ("Why did the 1987 Mars landing succeed?") invites the model to play along.

Types of hallucination

Fabricated facts Invented names, dates, quotes, citations.
Unfaithful Contradicts or adds to the source you gave.
Reasoning slips Wrong steps that look convincing.
Outdated Was true at training time, not now.

Each type needs a different defense.

1 / 2

Calibration and confidence signals

A model is calibrated if, when it is 80% sure, it is right about 80% of the time. Base models are often fairly well calibrated about their token probabilities, but chat models tuned for helpfulness tend to sound confident regardless. The tone of an answer is a weak signal of correctness; do not trust it.

Better signals:

  • Token probabilities. Some APIs return log-probabilities. Low probability on the tokens of a name or number is a warning sign.
  • Agreement across samples. Ask the same question several times at a temperature above zero. If the answers disagree, the model probably does not know. This is the idea behind consistency checks, shown below.
  • Verbalized confidence. Asking "how sure are you, 0 to 100?" helps a little but is noisy.
  • Retrieval support. Whether a supporting passage was found at all.

Defense 1: ground the answer in sources

The most effective fix is to stop asking the model to recall and instead give it the facts. In Retrieval-Augmented Generation you retrieve relevant passages and instruct the model to answer only from them, cite them, and say "I don't know" when the passages do not contain the answer. Answering with Context and Citations goes into the details. Grounding reduces fabricated facts but does not eliminate unfaithful answers, because the model can still misread or embellish the source.

Defense 2: verify

Treat model output as a draft to be checked:

  • Check against a source of truth in code. Does the cited URL resolve? Does the function exist? Is the product id in your database? Does the SQL parse?
  • Validate structure and ranges. Dates must be real, amounts non-negative, categories from a fixed list.
  • Run the code. For generated code, run tests rather than reading it.
  • Check quotes. Confirm each quoted passage actually appears in the source text.
  • Second pass. A separate call can check each claim against the sources ("Is this sentence supported by the text? yes/no"), as in the judge techniques of LLM-as-Judge and Human Review.
  • Human review where mistakes are costly.

Defense 3: prompt for honesty

Prompts cannot fix the underlying cause, but they help at the margin:

  • Give permission to abstain: "If you are not sure, say so. Do not guess."
  • Ask for evidence: "Quote the sentence that supports each claim."
  • Remove false premises and ask the model to check the premise first.
  • Keep the temperature low for factual tasks.
  • Use step-by-step reasoning for calculations, and tools such as a calculator or search for facts.

Self-consistency as a confidence signal

If a model really knows a fact, repeated samples agree. If it is guessing, they scatter. You can measure this: sample several answers, normalize them, and compute the share that matches the most common answer. Low agreement means "do not trust, verify or abstain".

Python A consistency checker with a fake model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The approach is simple and works for short factual answers. It costs several calls per question, and it cannot catch a model that is confidently and consistently wrong because of a training-data error. For long answers, split the text into individual claims and check each one.

Python Quote checking: is each citation really in the source?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A literal substring check is crude (paraphrases fail it) but it catches exactly the dangerous case: a "quote" that does not exist. Real systems combine it with a model that judges entailment.

A checklist

  1. Can the answer come from your own data? Retrieve it and cite it.
  2. Is a wrong answer costly? Add verification and human review.
  3. Can the model abstain? Give it permission and test that it uses it.
  4. Do you measure faithfulness on an eval set that includes unanswerable questions?
  5. Does the interface show sources, so users can check?

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do language models hallucinate?

    Next-token prediction rewards plausibility. Without a source, a missing fact still yields confident-sounding text.
  2. 2. Which approach most directly reduces fabricated facts in a company Q&A bot?

    Grounding replaces recall with supplied evidence, and citations let users verify.
  3. 3. Ten samples of the same question give ten different answers. What does that suggest?

    Low agreement across samples is a useful sign of uncertainty.
  4. 4. What is a weakness of self-consistency as a confidence signal?

    Agreement shows stability, not truth: a systematic error repeats in every sample.

Summary

  • Hallucinations come from next-token prediction, knowledge gaps, cutoffs and sampling; they will not disappear on their own.
  • A confident tone is not evidence of truth. Better signals are token probabilities, agreement between samples and supporting sources.
  • Ground answers in retrieved sources, require citations and allow "I don't know".
  • Verify in code (links, ids, quotes, tests) and add human review when mistakes are costly.
  • Measure faithfulness with eval cases that include unanswerable questions.

Next: handle hostile input in Prompt Injection and LLM Security.