Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 38 ยท Projects and Practice

Project: Grounded Question Answering

Build a small retrieval-augmented question answerer: chunk and index documents, retrieve the best passages, assemble a cited prompt, check grounding and handle unanswerable questions.

In this project you build the skeleton of a grounded question answering system: it answers only from a set of documents, cites where each fact came from, and says "I don't know" when the documents do not contain the answer. It combines retrieval-augmented generation, chunking, context assembly, guardrails and evaluation.

Everything runs in the editors. Retrieval uses a plain keyword-overlap score (TF-IDF style) so you can read every line; real systems use embeddings, but the pipeline shape is identical. The answering model is a stand-in that composes an answer from the top passage, which lets us test the surrounding system deterministically.

flowchart LR
    D["Documents"] --> C["Chunk"]
    C --> I["Index"]
    Q["Question"] --> R["Retrieve top k"]
    I --> R
    R --> A["Assemble prompt with sources"]
    A --> M["Model"]
    M --> G{"Grounding check"}
    G -- pass --> O["Answer with citations"]
    G -- fail --> N["I don't know"]

The grounding check and the "don't know" path are what make the system trustworthy.

Step 1: Documents, chunks and an index

Split each document into passages (here by paragraph), keeping the source id and chunk number so answers can cite them.

Python Chunk documents and build a TF-IDF index

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The question shares the word "refunds" with the returns chunk, so that chunk ranks first. A question with no overlapping word returns nothing, which we use below to detect "no evidence".

Step 2: Assemble the prompt

Put the rules first, then numbered sources, then the question. Tell the model to cite and to refuse when the sources are silent.

Python Build a prompt with numbered, citable sources

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 3: The pipeline with a grounding check

The check rejects answers that cite a source that was not provided, answers with no citation, and numbers that appear in no source. If retrieval found nothing, we skip the model entirely and say so.

Python End-to-end grounded QA with a stand-in model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last question gets "I don't know" because retrieval found no overlapping words. Read the other answers carefully, though: "Does shipping cost money?" quotes the delivery-time passage, and "What does the warranty exclude?" quotes the warranty length. Both are grounded (they really are in the documents) but neither answers the question. The stand-in only quotes the top passage; a real model would pick the relevant sentence, and the hit-rate test in step 4 tells you whether the right passage was even retrieved. Grounding is necessary, never sufficient. The grounding check is deliberately simple; stronger versions verify each claim against the cited text with an LLM judge or an entailment model.

Step 4: Evaluate retrieval and answers separately

When a RAG system answers wrongly, either retrieval missed the right passage or the model misused it. Measure them apart:

  • Retrieval hit rate: for each question with a known source, is the right chunk in the top k?
  • Answer correctness and faithfulness: is the answer right, and is every claim supported by the sources?
  • Abstention quality: does it say "I don't know" for unanswerable questions, and not for answerable ones?
Python Retrieval hit rate on a labelled set

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The misses show a known limit of keyword search: "money back" shares no word with "refunds", and "return window" does not match "returned" (no stemming). Typical fixes are covered in ranking and assembling context: add synonyms or query rewriting, combine keyword and embedding search, or retrieve more chunks.

Step 5: Harden it

  • Prompt injection: documents are untrusted data (defending). Wrap sources in delimiters, never give the model tools it does not need.
  • Access control: filter chunks by what the current user may see before retrieval, not after generation.
  • Freshness: re-index when documents change and show document dates in citations.
  • Cost: retrieve few, small chunks; cache repeated questions (caching).
  • Monitoring: log retrieved chunk ids, the "I don't know" rate and thumbs-down by question type (monitoring).

Extensions

  • Replace keyword scoring with embeddings and compare hit rates on your labelled set.
  • Add a chunk-overlap setting and measure its effect.
  • Return a quote with every citation and verify the quote appears in the source.
  • Add multi-turn support by rewriting follow-up questions into stand-alone ones.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does the pipeline skip the model when retrieval returns nothing?

    No retrieved evidence means no grounded answer is possible.
  2. 2. What does the grounding check reject?

    These are signs the answer is not supported by the supplied sources.
  3. 3. Why measure retrieval hit rate separately from answer quality?

    Separating the stages tells you whether to fix retrieval or the prompt.
  4. 4. A question about 'getting money back' misses the 'refunds' chunk. What is the likely cause and a fix?

    Lexical matching cannot connect different words with the same meaning.

Summary

  • Pipeline: chunk, index, retrieve, assemble a cited prompt, call, check grounding, abstain when unsure.
  • Number the sources and require citations so answers are checkable.
  • Evaluate retrieval (hit rate), answers (correctness, faithfulness) and abstention separately.
  • Harden with delimiters, access filtering before retrieval, freshness handling, caching and monitoring.

Next: Practice: Prompt Review Challenges