Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 36 ยท Architecture and Data Model

The Read Path: BlockCache and Bloom Filters

See how an HBase read merges the MemStore and several HFiles, how the BlockCache and bloom filters skip disk work, and how to tune reads for your workload.

A write in HBase is simple: log it and put it in memory. A read is more work, because the newest value of a cell might be in the MemStore, in the newest HFile, or in an older HFile, and deleted data may still be sitting in files. The RegionServer must combine all those sources into one correct answer, and do so in milliseconds. This lesson explains the pieces that make that possible.

You should know the write path first, because every read trick exists to cope with the way writes are stored.

Where a value can be

For one column family of one region, the sources of data are:

  1. The MemStore: the newest writes (and a snapshot being flushed).
  2. Several HFiles: from the newest flush back to old, compacted files.
  3. The BlockCache (not a source of truth, but a memory copy of HFile blocks).

A Get for a row builds a merge over a scanner on each source. Each scanner is positioned at the requested row, and a heap (priority queue) always yields the next cell in sorted order, newest version first. Delete markers are applied as cells flow past. The first cells that satisfy the request (for example "newest version of info:name") are returned.

flowchart LR
  G["Get / Scan"] --> MH["Merge heap"]
  MH --> MS["MemStore scanner"]
  MH --> F1["HFile 3 scanner"]
  MH --> F2["HFile 2 scanner"]
  MH --> F3["HFile 1 scanner"]
  F1 --> BC["BlockCache"]
  F2 --> BC
  F3 --> BC
  BC -.->|"miss"| D["HDFS block read"]

A read consults the MemStore and every HFile that might hold the row. The BlockCache avoids repeated disk reads.

Why many files slow reads

The more HFiles a Store has, the more scanners the heap must merge and the more blocks it may need to load. This is the real reason compaction matters for reads: fewer files means fewer places to look. Two features avoid unnecessary looks: indexes and bloom filters.

HFile indexes

Inside an HFile, a multi-level index points to the data block that contains a given key, so one lookup in a large file costs one block read (plus index blocks that are usually cached). Blocks are read and cached whole: the default block size is 64 KB. Smaller blocks (8 to 16 KB) help random point reads because less unneeded data is cached; larger blocks help long scans.

Bloom filters

A bloom filter is a compact data structure that answers "is this key definitely absent?" with no false negatives and a small number of false positives. HBase can store one per HFile. Before reading a file for a Get, HBase asks its bloom filter; if it says "not here", the file is skipped without touching disk.

Settings per column family (BLOOMFILTER):

  • NONE: no filter.
  • ROW (the default): filter on row key. Helps Gets and any read of whole rows.
  • ROWCOL: filter on row plus qualifier. Better when you fetch specific columns from wide rows, but larger.

Bloom filters do not help range scans, because a scan does not name an exact key. They help most when a row's data might be in one file out of many, which is exactly the common case after many flushes.

The stand-in below builds a real (simple) bloom filter with several hash functions, then uses it to skip files. It counts the files read with and without the filter.

Java Stand-in: a bloom filter skips files

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The BlockCache

The BlockCache keeps recently read HFile blocks in memory, shared by all regions on a server. The default implementation in HBase 2.x is CombinedBlockCache: an on-heap LRU cache (called LruBlockCache) for index and bloom blocks, plus an optional BucketCache that can live off-heap or on fast SSD storage for data blocks. Off-heap caching avoids Java garbage collection pauses for large caches.

  • By default the cache uses 40% of the heap (hfile.block.cache.size = 0.4).
  • The cache has priorities: in-memory column families (flag IN_MEMORY) are protected, regular blocks come next, and one-time scan blocks are the first to be evicted.
  • Large scans can evict useful blocks. Disable caching for them with scan.setCacheBlocks(false) (MapReduce jobs do this by default).
  • Reading the same keys again is fast because the block is already in memory; the first read pays the HDFS latency, including a network hop if the block is not local.

Short-circuit reads and locality

HBase prefers to run RegionServers on the same machines as the DataNodes. When a block replica is on the local disk, short-circuit local reads let the server read it directly from the file system, skipping the DataNode's network protocol. After a restart or a region move, locality can drop until compaction rewrites files locally; the metric hdfsBlocksLocalityIndex shows it.

A second stand-in: a small LRU block cache with hit/miss counting. Try changing the capacity to see how the hit ratio responds.

Java Stand-in: an LRU block cache

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The numbers show the "scan pollution" effect: the one-off scan pushes the hot blocks out of a small cache. That is why large scans should not use the block cache.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Where might the newest value of a cell be found during a read?

    The read merges the MemStore with all relevant HFiles and returns the newest visible version.
  2. 2. What does a bloom filter guarantee?

    A bloom filter has no false negatives; a false positive just causes an unnecessary file read.
  3. 3. Which workload benefits least from bloom filters?

    A scan does not name one key, so the filter cannot rule files out.
  4. 4. Why call scan.setCacheBlocks(false) for a huge full-table scan?

    One-pass scan blocks would flush useful data out of the cache.
  5. 5. Why does compaction help read performance?

    Fewer files mean fewer scanners to merge and fewer blocks to load.

Summary

  • A read merges the MemStore and all relevant HFiles through a heap of scanners, applying tombstones.
  • HFile indexes find the right block; blocks are cached whole (64 KB by default).
  • Bloom filters (ROW or ROWCOL) let a Get skip files that cannot contain the key.
  • The BlockCache (LRU on heap, optional BucketCache off heap) keeps hot blocks in memory; keep big scans out of it.
  • Fewer HFiles and good locality keep reads fast.

Next: Shell: Creating and Altering Tables