Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 53 ยท Retrieval-Augmented Generation (RAG)

Chunking and Preparing Documents

Prepare documents for retrieval: clean text, compare fixed-size, sentence and semantic chunking, choose overlap and chunk size, and attach useful metadata.

A model cannot read your 80-page manual in one prompt, and a search index cannot match a question to "the manual" as a whole. So before retrieval, documents are cut into chunks: passages small enough to embed and paste into a prompt, yet large enough to make sense on their own.

Chunking looks boring, but it is one of the biggest levers on RAG quality. Cut in the wrong place and the answer is split across two chunks, neither of which retrieves well. This lesson covers cleaning, three chunking strategies, overlap, metadata and how to pick a size.

Clean first

Text from PDFs, web pages and slides is messy. Garbage in the index becomes garbage in the prompt. Typical cleanup:

  • Remove navigation, cookie banners, repeated headers and footers, page numbers.
  • Fix hyphenation and broken line wrapping from PDFs (inter-\nnational).
  • Normalize whitespace and Unicode quotes, but keep paragraph breaks: they are valuable boundaries.
  • Keep structure that carries meaning: headings, list items and table rows. Convert tables to a readable text form rather than letting columns scramble.
  • Drop exact duplicates, which would otherwise fill your top results with copies.
Python A small text cleaner

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice what stays: the blank line between paragraphs. Cleaning should remove noise without destroying the boundaries that chunking relies on.

Three chunking strategies

Pick a cutting rule

Fixed size Every N characters or tokens. Simple, may cut mid-sentence.
Sentence based Group whole sentences up to a size limit.
Semantic Cut where the topic changes, using headings or similarity.

Start simple; move to structure-aware only if tests show a need.

1 / 2

Fixed-size chunking slices the text every N characters (or tokens). It is fast and predictable, but happily cuts a sentence, a word or a code block in half.

Sentence-based chunking splits into sentences first, then packs whole sentences into a chunk until a size limit. No sentence is ever torn apart.

Semantic or structure-aware chunking cuts at meaningful boundaries: headings, paragraphs, list items, function definitions in code, or points where adjacent sentences stop being similar (computed with embeddings). It produces coherent chunks at the cost of more code and uneven sizes.

The example implements all three on one short text so you can compare them. The semantic version here uses a simple stand-in for "topic change": a new paragraph begins a new chunk.

Python Three chunkers side by side

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the fixed-size output you can see chunks that begin or end mid-word. The paragraph version keeps each topic (warranty, returns, batteries) together, but chunk sizes are uneven. Real splitters often combine rules: split on paragraphs first, and only if a paragraph is too long, fall back to sentences, then to characters. This is called recursive splitting.

Overlap

When you cut text, information near the cut can end up split: "The refund" at the end of one chunk and "takes 5 days" at the start of the next. Overlap repeats the last part of each chunk at the beginning of the next, so a passage that straddles a boundary still appears whole in at least one chunk.

Typical overlap is 10 to 20 percent of the chunk size. More overlap means better coverage but more storage, more embeddings to compute and near-duplicate results in the top list.

Python Fixed windows with overlap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With zero overlap, the sentence is cut between "five days" and "after"; with an overlap of 3 words, the second chunk still begins with the end of the first, so a question about timing finds the whole idea in one place.

Metadata

A chunk should travel with information about where it came from:

  • Source: document title, URL or file path, so you can cite it.
  • Location: page number, section heading or chunk index.
  • Dates: created and updated, so you can prefer fresh material.
  • Access: which users or groups may see it, so retrieval can filter (see Vector Retrieval in Practice).
  • Type or language: to filter "only FAQs" or "only German".

A useful trick is to prepend context to the chunk text before embedding, such as the document title and heading ("Warranty > Water damage: ..."). A chunk that says only "It does not cover water damage" is ambiguous on its own; with its heading attached, the embedding knows what "it" is.

Python Chunks with metadata

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Choosing a chunk size

There is no universal best size, only trade-offs:

Smaller chunks Larger chunks
Precise matches; embeddings represent one idea More context per hit; fewer split answers
Many chunks, bigger index Fewer chunks, cheaper index
Risk of losing surrounding context A vector blurs several topics together
More chunks fit in the prompt budget Fewer fit, and unrelated text wastes tokens

Reasonable starting points are roughly 200 to 500 tokens with 10 to 20 percent overlap, but treat that as a guess. Measure it: build a set of questions with known source passages, try a few sizes, and see how often the right chunk lands in the top results (see Metrics: Retrieval, Benchmarks and Statistics). Respect the limits of your embedding model, too: input beyond its maximum length is silently cut off.

Another common design is parent-child chunking: search over small chunks for precision, but hand the model the larger surrounding section for context.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a downside of fixed-size chunking?

    Fixed windows ignore the meaning of the text, so boundaries may fall mid-sentence.
  2. 2. What does overlap between chunks achieve?

    Repeating the tail of one chunk at the head of the next protects information near the cut, at the cost of more storage.
  3. 3. Why prepend the document title and heading to a chunk before embedding?

    Context in the text makes the embedding, and the final prompt, more specific and less ambiguous.
  4. 4. How should you decide the best chunk size for your data?

    The right size depends on your documents and questions, so measure retrieval quality instead of guessing.

Summary

  • Clean documents first, but keep paragraph, heading and list boundaries.
  • Fixed-size chunking is simple; sentence and structure-aware chunking respect meaning; recursive splitters combine them.
  • Overlap of roughly 10 to 20 percent protects ideas near boundaries.
  • Store metadata (source, section, page, dates, access) and consider prepending headings to the embedded text.
  • Smaller chunks match precisely, larger ones keep context; choose by measuring retrieval quality.

Next: Vector Retrieval in Practice.