Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 36 ยท Generation and Prompting

Advanced RAG: HyDE, Parent Documents, Corrective and Agentic RAG

Go beyond basic RAG: hypothetical document embeddings, parent-document retrieval, query decomposition, corrective retrieval loops and agentic RAG, with code.

A basic RAG pipeline is embed, retrieve, stuff, answer. It works surprisingly well, and you should always build and evaluate that baseline first (evaluation chapter). When measurements show specific failures, a handful of well-known patterns address them. This lesson surveys the ones that pay off most, with runnable miniatures of each. Language-model calls are replaced by tiny stand-ins so you can see the control flow without a network.

Add complexity only when your evaluation set shows the baseline failing in a way the pattern fixes. Each pattern adds latency, cost and new failure modes.

Pattern map

Which pattern fixes what?

Parent-child Small chunks to match, big ones to read
HyDE Search with a draft answer
Decompose Split multi-part questions
Corrective Check and retry retrieval

Pick by the failure you measured, not by fashion.

1 / 2

Parent-document (small-to-big) retrieval

Small chunks embed precisely (one idea each) but give the model too little context. Big chunks give context but embed vaguely. Small-to-big gets both: index small child chunks, and when a child matches, return its larger parent (the surrounding section or page).

Python Match on sentences, return the section

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The sentence "Gift cards are not refundable" is the precise match; the model receives the whole refund section, including the pro rata rule it may need for context. Variants: sentence-window retrieval (return the matched sentence plus N neighbours), and hierarchical indexes that summarize sections.

HyDE: Hypothetical Document Embeddings

A question and its answer often look different ("why is my invoice higher?" versus a paragraph about proration). HyDE asks an LLM to write a short hypothetical answer, embeds that, and searches with it. The draft may contain factual errors, but it uses the vocabulary and shape of real answers, so it lands near real answer chunks. It helps most on short, vague or question-style queries against document-style corpora.

Python HyDE with a hand-written hypothetical answer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The raw question shares no words with the proration chunk, so its best score is zero; the draft shares several. Cost: one extra generation per query. Risk: if the model hallucinates in the wrong direction, retrieval follows it, so many teams combine the HyDE query with the original via RRF.

Decomposition and step-back

  • Decomposition: "Compare the Team and Business plans on seats and support" becomes sub-questions about Team and about Business; retrieve for each, merge the evidence, then answer once.
  • Step-back: ask a more general question first ("What do the plans include?") to fetch background, then the specific one. Helps when the specific question is narrow and the answer needs general rules.
  • Sequential (multi-hop): the answer to step 1 feeds step 2 ("Who is the account owner, and what is their plan?"), so the steps cannot run in parallel.

Corrective RAG: check, then retry

Instead of trusting whatever retrieval returns, grade it. If the evidence looks weak, change strategy: rewrite the query, widen the search, or fall back to another source (web search, human). The grader can be a score threshold, a cross-encoder, or an LLM judging relevance.

Python A corrective retrieval loop

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Self-RAG and CRAG are research variants of the same loop: the model emits "reflection" judgments (is retrieval needed, is this passage relevant, is my answer supported) and the system acts on them.

Agentic RAG

In agentic RAG the model is given tools, such as search_docs(query, filters), get_page(id) or run_sql(query), and runs a loop: think, call a tool, read the result, decide whether to search again or answer. It handles multi-step and multi-source questions well. The costs are real: more model calls, higher latency, less predictable behavior, and a larger attack surface (security lesson). Set a hard cap on steps and tokens, log every tool call, and evaluate on full question-to-answer trajectories.

Graph RAG and other structured retrieval

When answers depend on relationships (who reports to whom, which service depends on which), plain similarity struggles. Graph RAG extracts entities and relations into a knowledge graph, retrieves a relevant subgraph or community summary, and passes that to the model. It shines for corpus-wide questions ("what are the main themes?") and multi-hop reasoning, but extraction is expensive and needs maintenance. Related options: SQL or API tools for structured data, and table-aware chunking for spreadsheets.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does small-to-big retrieval do?

    Small chunks give precise matches; the parent supplies the context the model needs.
  2. 2. In HyDE, what is embedded for the search?

    The draft uses answer-like vocabulary, so it lands near real answer chunks.
  3. 3. What is the core idea of corrective RAG?

    A grader decides whether to answer, rewrite, widen the search or fall back.
  4. 4. Which is a real risk of agentic RAG?

    Agent loops cost more and vary more, so cap steps and log tool calls.

Summary

  • Start with a baseline; add a pattern only to fix a measured failure.
  • Small-to-big separates what you match on from what you read.
  • HyDE, multi-query and decomposition improve queries; fuse results with RRF.
  • Corrective loops grade evidence and retry or fall back.
  • Agentic and graph approaches handle multi-step and relational questions at higher cost.

Next: Retrieval Metrics: Recall, MRR and nDCG