Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 36 ยท Getting Started

What Is Retrieval-Augmented Generation?

Learn what retrieval-augmented generation (RAG) is, why language models need it, how the retrieve-then-generate pipeline works, and build a tiny one in Python.

A large language model (LLM) knows a lot, but it knows it the way you know a book you read years ago: blurry, frozen in time, and impossible to look up. It has never seen your company wiki, last week's release notes or your customer's contract. Ask about those and it will either say "I don't know" or, worse, invent a confident answer.

Retrieval-augmented generation (RAG) fixes this by splitting the job in two. First a retriever finds the few passages from your own documents that are relevant to the question. Then a generator (the LLM) writes the answer while reading exactly those passages. The model no longer has to remember the facts. It only has to read and explain them.

This track teaches the whole path: vectors and embeddings, vector databases and indexes, chunking, retrieval, prompting, evaluation and production. In this first lesson you will see the idea end to end with the smallest possible pieces.

What runs here. The code in this track runs for real in your browser with Python, NumPy and scikit-learn. No paid LLM or embedding API runs on this site, so the "generator" in our examples is a small stand-in function and the "embeddings" are built from word statistics. Everything that depends on a real hosted model (OpenAI, Cohere, Sentence-Transformers, and so on) is shown as a read-only snippet and clearly labelled.

Why a plain LLM is not enough

There are four recurring problems:

  1. Stale knowledge. A model's training data stops at a cutoff date.
  2. Private knowledge. Internal documents were never in the training data.
  3. Hallucination. When the model lacks the fact, it still produces fluent text.
  4. No sources. You cannot ask a model "where did you read that?" and get a trustworthy answer.

You could fine-tune the model on your documents, but fine-tuning is slow, expensive, hard to update and does not give citations. RAG is the cheap alternative: change the documents and the answers change immediately.

The pipeline in one picture

flowchart LR
    D["Documents"] --> C["Chunk"]
    C --> E["Embed"]
    E --> V[("Vector store")]
    Q["User question"] --> QE["Embed question"]
    QE --> S["Search top-k"]
    V --> S
    S --> P["Build prompt<br/>(question + passages)"]
    P --> L["LLM"]
    L --> A["Answer with sources"]

The top row runs once, ahead of time (ingestion). The bottom row runs on every question (query time).

Two phases

Ingest Split documents, embed, store.
Query Embed the question, search, answer.

Ingestion is offline. Query time must be fast.

1 / 2

A tiny RAG system with no libraries

Here is the whole idea in about thirty lines. The knowledge base is a list of short passages. The retriever scores each passage by how many question words it shares. The "generator" is a stand-in, because we have no LLM here, so we only print the prompt that a real LLM would receive.

Python Retrieve, then build the prompt

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at what the LLM would receive: instructions, numbered passages and the question. That is all RAG is at its core. Everything else in this track (embeddings, indexes, chunking, rerankers, evaluation) exists to make the "retrieve" box return better passages, faster, at scale.

Where keyword matching breaks

Our retriever counted shared words. That fails as soon as people phrase things differently from the documents.

Python The vocabulary mismatch problem

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The refund document never contains the word "refund", so keyword overlap sees no real connection. A human knows "return a purchase to get money back" means a refund. Embeddings capture that meaning as numbers, and vector databases search those numbers quickly. The next lessons build exactly that.

RAG compared with the alternatives

Approach Good at Weak at
Prompt only Quick demos, general knowledge Private or fresh facts
Fine-tuning Style, format, specialised skills Facts that change; citations
Long context (paste everything) Small document sets Cost and latency; hundreds of pages; noisy
RAG Large, changing, private knowledge with sources Needs good retrieval; extra moving parts

Often you combine them: a fine-tuned model for tone, RAG for facts.

What can go wrong

  • Bad retrieval. If the right passage is not retrieved, the best LLM cannot answer. This is the most common failure.
  • Bad chunks. Passages cut in the wrong place lose their meaning (chunking).
  • Ignored context. A model may answer from memory despite the passages, so the prompt must be firm.
  • Stale index. If documents change and the index does not, answers are out of date (ingestion).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What are the two main components of a RAG system?

    The retriever finds relevant passages and the generator (an LLM) writes the answer from them.
  2. 2. Which problem does RAG help with most directly?

    RAG injects up-to-date, private documents into the prompt, so the model does not need to have memorised them.
  3. 3. Why does simple keyword overlap often fail as a retriever?

    This is the vocabulary mismatch problem. Embeddings compare meaning instead of exact words.
  4. 4. Which phase runs offline, before any question is asked?

    Documents are processed once ahead of time; only the query-side steps run per question.

Summary

  • RAG = retrieve relevant passages, then let an LLM answer from them.
  • It gives fresh, private, citable answers without retraining the model.
  • Ingestion is offline; query time embeds, searches, builds a prompt and generates.
  • Keyword matching misses synonyms; embeddings and vector search fix that.
  • Retrieval quality is usually the limiting factor.

Next: What Is a Vector Database?