Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 36 ยท Administration and Tuning

Region Splitting and Balancing

How HBase splits regions as tables grow, split policies, manual and pre-splitting, merging, the balancer and region server assignment, with a runnable Java model.

A table in HBase is divided into regions: contiguous ranges of row keys, each served by exactly one region server at a time. Regions are the unit of distribution and of failover. This lesson covers how regions are created, why and when they split, how the balancer spreads them over servers, and what you can tune.

HBase cannot run in the live editors here. Shell commands and configuration are shown read-only. The runnable blocks are Java models of the splitting and balancing logic, not HBase code.

Region lifecycle

A new table without split keys has one region covering the whole key space. As data is written, the region grows. When a store file in it becomes too large, the region server splits it into two daughter regions, each taking half of the key range. The parent is closed, the daughters open, and the Master updates hbase:meta, the system table that maps key ranges to servers. Clients look regions up in hbase:meta (through a cached location) so they talk directly to the right region server.

flowchart LR
  P["Region [a, z)<br/>10 GB"] -->|"split at a midpoint row"| D1["Daughter [a, m)"]
  P -->|"split"| D2["Daughter [m, z)"]
  D1 --> RS1["Region server 1"]
  D2 --> RS2["Region server 2"]

Daughters initially share the parent's files through reference files; compaction rewrites them into real files.

Split policies

The policy decides when to split. Set it per table (SPLIT_POLICY) or in hbase.regionserver.region.split.policy.

  • SteppingSplitPolicy (the default in HBase 2.x): a table with a single region splits early (at 2 x the flush size, so around 256 MB with the default flush), then later regions split at hbase.hregion.max.filesize (default 10 GB). Small new tables get spread out quickly.
  • IncreasingToUpperBoundRegionSplitPolicy: the split threshold grows with the number of regions of the table on the server.
  • ConstantSizeRegionSplitPolicy: split only at max.filesize.
  • KeyPrefixRegionSplitPolicy and DelimitedKeyPrefixRegionSplitPolicy: never split in the middle of a key prefix, which keeps all rows of one entity in a single region.
  • DisabledRegionSplitPolicy: no automatic splits; you manage them by hand.

The split point is the middle row of the largest store file. HBase cannot split a single row: a gigantic row stays in one region.

Pre-splitting versus auto-splitting

Auto-splitting is fine for general tables, but it has downsides for write-heavy tables with known key distribution: early writes hit one region (see hotspotting), splits happen under load, and splitting causes brief unavailability of the region. Pre-splitting creates the table with split points chosen from the key design:

TEXT
create 'events', 'd', SPLITS => ['2','4','6','8','a','c','e']
create 'events', 'd', {NUMREGIONS => 16, SPLITALGO => 'HexStringSplit'}

You can also split a live region by hand: split 'events' (all regions) or split 'region-name', 'split-row'. Some teams combine pre-splitting with DisabledRegionSplitPolicy to avoid automatic splits entirely and handle growth with planned splits.

Rule of thumb for region count: keep the total regions per server in the tens to low hundreds (roughly 20 to 200), and aim for regions around 5 to 20 GB. Thousands of tiny regions per server waste MemStore memory and slow the Master.

Merging

Regions that became small or empty (for example after TTL expiry on old time ranges) can be merged to reduce overhead: merge_region 'ENCODED_NAME_A', 'ENCODED_NAME_B'. The two regions must be adjacent. HBase 2.x also has a normalizer (normalize, normalizer_enable) that proposes splits and merges to even out region sizes within a table.

Assignment and the balancer

The Master assigns regions to region servers. When servers join, leave or crash, regions move. The balancer is a periodic Master job (hbase.balancer.period, default 5 minutes) that moves regions so that each server holds a similar number. The default StochasticLoadBalancer weighs several costs: region count, read and write request load, store file size, data locality (how much of a region's data lives on the same HDFS DataNode) and moves needed.

Useful shell commands:

TEXT
balance_switch true       # enable the balancer
balancer                  # run it now
move 'ENCODED_REGION_NAME', 'host,port,startcode'
assign 'REGION_NAME'      # assign an offline region
unassign 'REGION_NAME'

Moving a region closes it and reopens it elsewhere, so it is briefly unavailable and has to warm its caches; locality may be poor until the next major compaction rewrites its files on the new server's local DataNode. That is one reason not to run the balancer aggressively during peak hours.

A model of splitting and balancing

The program simulates a table that starts with one region and splits at a size limit, then balances region counts over three servers by moving regions from the busiest server to the least busy.

Java Splits and balancer (simulation)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

All daughters begin on the server that held the parent, so with sequential keys rs1 ends up with every region until the balancer moves some. This is why a single hot server persists unless the key design spreads writes.

Practical advice

  • Choose pre-splits from the real key distribution, then check region sizes and request counts per region in the Master UI.
  • Prefer fewer, larger regions (within reason) to thousands of small ones.
  • Watch region-in-transition counts: regions stuck in transition after a crash need attention (hbck2 tooling in HBase 2).
  • Keep the balancer on, but let it work with a throttle and off-peak schedule on large clusters.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does HBase do when a region grows past the split threshold?

    The region server splits the region, the daughters take halves of the key range, and hbase:meta is updated.
  2. 2. Why pre-split a new write-heavy table?

    A table starts with one region; pre-splitting distributes writes across servers from the start.
  3. 3. What does the balancer do?

    The balancer reassigns regions so servers hold similar load, weighing counts, request load and locality.
  4. 4. What happens to a very large single row?

    Region boundaries fall between rows, so one huge row remains in a single region.

Summary

  • Regions are key ranges served by one server; they split as they grow and are tracked in hbase:meta.
  • SteppingSplitPolicy is the default; prefix policies keep related rows together.
  • Pre-split write-heavy tables and keep region counts moderate.
  • The balancer moves regions to even load, at the cost of brief unavailability and lost locality.

Next: MemStore, Block Cache and Bloom Filters