Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 35 ยท Architecture and Setup

Architecture: Parts, Granules and Merges

Understand how ClickHouse stores data: immutable parts, column files, granules and marks, background merges, and how a query is executed in parallel.

To use ClickHouse well you need a small mental model of what happens on disk when you insert and when you query. The model has four words: part, granule, mark and merge. Once you know them, advice like "insert in batches", "choose the sorting key carefully" and "do not over-partition" stops being a list of rules and becomes obvious.

ClickHouse cannot run on this site. The system-table queries are read-only. The runnable blocks simulate the idea (grouping rows into granules, finding the granules a filter needs) in SQLite and say so.

Every insert creates a part

When you run an INSERT, ClickHouse takes the batch, sorts it by the table's ORDER BY key, splits it into columns, compresses each column and writes a directory called a part. Parts are immutable: ClickHouse never edits one in place. That makes writes fast and safe, and it is why single-row updates are costly.

A part directory looks like this (names simplified):

TEXT
all_1_1_0/
  primary.idx        sparse primary index
  checksums.txt
  columns.txt
  count.txt
  user_id.bin        compressed data of one column
  user_id.mrk3       marks: where each granule starts in the .bin file
  page.bin
  page.mrk3
  ...

The directory name all_1_1_0 means partition all, block range 1 to 1, merge level 0. You can list parts with:

SQL
SELECT name, rows, bytes_on_disk, level
FROM system.parts
WHERE table = 'page_views' AND active;

Granules and marks

Inside a part the sorted rows are grouped into granules, by default 8192 rows each (index_granularity). The sparse primary index stores one entry per granule, not per row: the sorting-key values of the first row of each granule. A mark file tells ClickHouse where in each column's .bin file a granule begins.

Why not index every row? An index entry per row would be enormous. One entry per 8192 rows means a billion-row table has about 122,000 index entries, small enough to sit in memory.

From insert to part

INSERT batch Rows arrive
Sort By the ORDER BY key
Split One file per column
Write part Immutable directory

Every insert makes one new part.

1 / 2

How a query uses granules

For WHERE page = '/pricing', ClickHouse binary-searches the sparse index to find which granules might contain /pricing, then reads only those granules, only from the columns it needs. Everything else is skipped without being read. A query that cannot use the key (a filter on an unsorted column) must read all granules, which is a full scan. It is still fast, but not free. The details are in MergeTree ORDER BY and the primary key.

You can imitate the idea in SQL. Below, rows sorted by page are numbered into granules of 3 rows, and we compute which granules a filter touches using each granule's first and last key.

SQL Stand-in: which granules does a filter need? (SQLite)

Press Run (or Ctrl + Enter) to see the result here.

SQL Stand-in: granules to read for page = '/d'

Press Run (or Ctrl + Enter) to see the result here.

Only 1 of 4 granules overlaps /d here. In real tables with millions of granules the fraction can be tiny.

Background merges

Many small parts hurt: each query must open and merge results from all of them. So ClickHouse runs background merges that combine several parts into one larger part (re-sorting as it goes). Merge results keep growing, and old parts are deleted after a delay. That is the "Merge" in MergeTree.

flowchart LR
    A["Insert 1<br/>part 1_1_0"] --> M["Background merge"]
    B["Insert 2<br/>part 2_2_0"] --> M
    C["Insert 3<br/>part 3_3_0"] --> M
    M --> D["Merged part<br/>1_3_1"]

Three small parts become one larger part at merge level 1.

Merges are also where special engines do their work: ReplacingMergeTree removes duplicate rows, SummingMergeTree adds numbers and AggregatingMergeTree combines states during merges. Because merges happen eventually, not instantly, those engines show their effect "eventually". See replacing and collapsing engines.

If you insert too fast and make parts faster than merges can combine them, ClickHouse first slows inserts and finally refuses with Too many parts. The cure is bigger batches, not more inserts.

Wide and compact parts

Parts with many rows are stored as wide parts (a file per column). Very small parts may be stored compact (all columns in one file). You will meet this only in settings and system.parts output. Query behaviour is the same.

Parallel, vectorized execution

A query is turned into a pipeline: read granules, filter, aggregate. ClickHouse runs the pipeline on many threads, each reading different granules, and moves blocks of rows (around 65,000) through each step with vectorized code. A GROUP BY first aggregates per thread, then merges the partial results. With several shards the same idea applies across servers (the later chapters on replication and clusters cover this).

To see what a query will do:

SQL
EXPLAIN indexes = 1
SELECT count() FROM page_views WHERE page = '/pricing';

The output lists how many parts and granules the primary key kept out of the total. That number is the quickest check that your key works.

Async inserts

If many clients each send small inserts, you can ask the server to buffer and batch them for you:

SQL
INSERT INTO page_views SETTINGS async_insert = 1, wait_for_async_insert = 1
VALUES ('2026-05-01 09:00:01', 1, '/home', 'PT', 120);

The server collects rows from many requests and writes them as one part. It trades a little latency for far fewer parts.

Common beginner mistakes

  • Many tiny inserts and a "Too many parts" error.
  • Over-partitioning, which multiplies parts (see partitioning and parts).
  • Assuming a merge happens right after an insert. It is background and eventual.
  • Expecting OPTIMIZE TABLE to be a routine tool; it forces a merge and is expensive on big tables.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a part in ClickHouse?

    Every insert creates a part; merges create bigger parts from smaller ones.
  2. 2. By default, how many rows are in a granule?

    index_granularity defaults to 8192 rows; the sparse index has one entry per granule.
  3. 3. Why are background merges needed?

    Merging parts keeps reads efficient and is where ReplacingMergeTree and others do their work.
  4. 4. What does a 'Too many parts' error usually mean?

    The cure is fewer, larger inserts or async inserts, not more of them.

Summary

  • An insert writes an immutable, sorted, compressed part; parts never change in place.
  • Rows in a part form granules (8192 rows); the sparse index and marks point at granules.
  • Queries skip granules that cannot match the key, and read only the needed columns.
  • Background merges combine parts and carry out engine-specific logic.
  • Use big batches or async inserts to avoid "Too many parts".

Next: Install and run ClickHouse