Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 36 ยท Administration and Tuning

MemStore, Block Cache and Bloom Filters

Tune the HBase read path: BlockCache and BucketCache, MemStore sizing, Bloom filter types, block size and compression, with runnable LRU and Bloom filter models.

Reads in HBase combine three places: the MemStore (recent writes), the BlockCache (recently read HFile blocks in memory) and the HFiles on disk. A read that is served from memory takes well under a millisecond; one that goes to HDFS can take many milliseconds. Most HBase tuning is about making the right data live in the right memory region and avoiding useless disk reads.

This lesson covers memory split, the block cache, Bloom filters, block size and compression. It follows Compactions and Flushes.

HBase cannot run in the live editors. Configuration keys and shell commands are read-only. The Java blocks are runnable models of an LRU cache and a Bloom filter, labelled as such.

Where the heap goes

A region server heap is shared by two big consumers:

  • MemStores: up to hbase.regionserver.global.memstore.size of the heap (default 0.4).
  • BlockCache: up to hfile.block.cache.size of the heap (default 0.4).

The two together must stay below about 80% of the heap (HBase refuses to start otherwise), leaving the rest for everything else. A write-heavy workload should give more to the MemStore; a read-heavy one more to the cache. Rarely do both need much. Since a block cache hit is the cheapest way to serve a read, read-heavy clusters often move the cache off-heap.

BlockCache: LRU and BucketCache

HFiles are divided into blocks (default 64 KB). A read loads the needed block from disk and stores it in the cache. Two implementations:

  • LruBlockCache (on-heap): an LRU with three priorities. Single access blocks (read once) get 25% of the cache, multi access blocks (read again) 50%, and in-memory blocks 25%. This makes scan-once data less likely to evict hot data.
  • BucketCache: stores blocks in off-heap memory or on fast local SSD files. The default production setup (CombinedBlockCache) keeps index and Bloom blocks in the on-heap LRU and data blocks in the BucketCache, which avoids long Java garbage-collection pauses on big caches.

Per column family settings:

TEXT
create 't', {NAME => 'd', IN_MEMORY => 'true', BLOCKCACHE => 'true'}

IN_MEMORY gives the family's blocks the highest priority. It does not pin them forever; it only makes eviction less likely. Per scan, setCacheBlocks(false) keeps a one-time full scan or MapReduce job from flushing the hot working set out of the cache.

Java An LRU block cache (model)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Only reads of the hot set are counted. The middle run loses its hot blocks to the scan and pays 5 extra misses afterwards (90% instead of 95%); the third run keeps the cache intact. With a longer-lived workload the loss repeats with every big scan. (Real LruBlockCache also has the three priority groups, which soften this effect; the model uses plain LRU to show the idea.)

Bloom filters

A Bloom filter is a compact probabilistic set: it can say "definitely not in this file" or "maybe in this file". HBase stores one per HFile (or per block). Before reading a file for a Get, the region server asks the filter; a "no" skips the file entirely. Bloom filters make point reads on tables with many HFiles much cheaper, at the cost of a little space and write-time CPU.

Types, per family (BLOOMFILTER):

  • NONE
  • ROW (default): test by row key.
  • ROWCOL: test by row key plus column qualifier; helps Gets for specific columns in wide rows.
  • ROWPREFIX_FIXED_LENGTH: test by a fixed-length key prefix, useful for prefix scans and tables with a known entity-id prefix.

Bloom filters help point lookups and some prefix scans. They do not help range scans over unknown keys.

Java A Bloom filter (model)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

There are never false negatives, so a Bloom filter never makes you miss existing data. The false-positive rate is controlled by io.storefile.bloom.error.rate (default 1%).

Block size, encoding and compression

  • Block size (BLOCKSIZE, default 64 KB): smaller blocks (8 to 16 KB) suit random Gets (less data read per lookup, larger index); larger blocks (128 to 256 KB) suit sequential scans.
  • Data block encoding (DATA_BLOCK_ENCODING => 'FAST_DIFF' or PREFIX/DIFF): stores each key relative to the previous one, which saves a lot of space when many cells share a long row key and qualifier. It usually helps both disk and cache, at small CPU cost.
  • Compression (COMPRESSION => 'SNAPPY', LZ4, ZSTD, GZ): shrinks HFiles on disk; blocks are decompressed when loaded into the cache. SNAPPY or LZ4 are common for speed; ZSTD for a better ratio.
TEXT
alter 'events', {NAME => 'd', COMPRESSION => 'ZSTD', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', BLOCKSIZE => '32768'}

Existing files only pick up new settings after they are rewritten, by a major compaction.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which two consumers share most of a region server heap?

    By default each may use about 40% of the heap, and together they must stay under about 80%.
  2. 2. A Bloom filter says "no" for a row in an HFile. What does that guarantee?

    Bloom filters have no false negatives, so a no means the file can be skipped.
  3. 3. Why call setCacheBlocks(false) on a large one-off scan?

    A big scan would otherwise flood the cache with blocks that are never read again.
  4. 4. When do new COMPRESSION or BLOCKSIZE settings affect existing data?

    Settings apply to newly written HFiles; compaction rewrites old ones.

Summary

  • Heap is split between MemStores and the BlockCache; read-heavy clusters often use an off-heap BucketCache.
  • The BlockCache is an LRU of HFile blocks; keep one-off scans out of it.
  • Bloom filters (ROW, ROWCOL, prefix) let Gets skip HFiles; they never miss existing rows.
  • Block size, data block encoding and compression trade disk, CPU and latency.

Next: Backup, Snapshots and Replication