Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 36 ยท Getting Started

Your First RAG Pipeline

Build a complete retrieve-and-answer RAG pipeline in Python: chunk documents, embed with TF-IDF vectors, search by cosine similarity, build the prompt and cite sources.

In this lesson you will build a working RAG pipeline end to end: ingest documents, search them with vectors, assemble a prompt and return an answer with sources. To keep it runnable in your browser, we use TF-IDF vectors (a classic statistical embedding) and a stand-in generator instead of a hosted model. The structure is identical to a production system; only two components get swapped later.

The five steps

Chunk Split documents into passages.
Embed Turn each passage into a vector.
Store Keep vectors and text together.
Retrieve Find the top-k passages.

Then build the prompt and call the LLM.

1 / 2

Step 1: the documents

Our corpus is a small help center for an imaginary app. Each document has an id and a title so we can cite it.

We define it in full in the runnable example below: six short help-center articles keyed by id.

Step 2: chunk, embed and store

We treat every document as one chunk for now (real chunking comes in chapter 4). TfidfVectorizer turns each chunk into a vector whose length equals the vocabulary size. It is sparse (mostly zeros), but cosine similarity works the same way.

Step 3: retrieve

Embed the question with the same vectorizer, then rank chunks by cosine similarity. Using the same model for documents and queries is a hard rule: vectors from different models live in different spaces and cannot be compared.

Step 4: build the prompt and answer

The prompt contains instructions, numbered passages and the question. The stand-in generator returns the best sentence from the top passage, which is enough to demonstrate grounding and citations.

Python A complete mini RAG pipeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Three behaviours are worth noticing:

  • The first two questions find the right document and cite it.
  • The third question retrieves nothing above min_score, so the pipeline refuses instead of guessing. A similarity threshold is one of the cheapest defenses against hallucination.
  • prompt() shows exactly what a real LLM would receive.

Swapping in a real LLM (read-only)

The only change to call a hosted model is to send the prompt to an API. This snippet is read-only: no model API runs on this site.

Python
# Read-only illustration: requires an API key and network access.
from openai import OpenAI

client = OpenAI()
hits = rag.retrieve(question, k=3)
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": rag.prompt(question, hits)}],
    temperature=0,
)
print(response.choices[0].message.content)

Any provider works the same way: build a prompt string (or a list of messages), send it, read the text. Low temperature makes answers more faithful to the context.

Seeing the retrieval

Print the whole similarity table once. It teaches you what the retriever "thinks" and is the first debugging tool you should reach for.

Python Inspect the similarity scores

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the third question: "are my notes safe?" scores only 0.32 against the security document, a thin margin over 0.22 for the trash document. The match comes from the shared word "notes", not from understanding that "safe" relates to "encrypted". That is TF-IDF's limit. Neural embeddings, covered in embedding models, know that "safe" and "encrypted" are related.

Pipeline checklist

When your own pipeline misbehaves, check in this order:

  1. Did ingestion include the document at all?
  2. Is the right passage in the top-k? (If not, it is a retrieval problem.)
  3. Is the right passage in the prompt, uncut?
  4. Did the model use it? (If not, it is a prompting problem.)

Most "the LLM is wrong" bugs are actually steps 1 to 3.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why must documents and queries be embedded with the same model?

    Similarity only makes sense inside one embedding space. Mixing models gives meaningless scores.
  2. 2. What does the min_score threshold do in the example?

    If no passage clears the threshold, the pipeline answers that it does not know instead of letting a model guess.
  3. 3. The correct passage is not in the top-k results. Where is the bug?

    The generator can only use what it is given. Fix retrieval before touching the prompt.
  4. 4. Which part of the example is a stand-in for something real?

    No LLM runs here, so a tiny extractive function replaces it. The retrieval parts are real.

Summary

  • A RAG pipeline is chunk, embed, store, retrieve, prompt, generate.
  • Use the same embedding model for documents and queries.
  • A similarity threshold lets the system say "I do not know".
  • Print the scores: retrieval problems are visible there.
  • Calling a real LLM only changes the last step.

Next: The Vector Database Landscape