Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 36 ยท Architecture and Data Model

Tables, Rows, Column Families and Qualifiers

Understand the HBase data model in depth: namespaces, tables, row keys, column families, qualifiers and the sorted, sparse layout that makes scans fast.

The HBase data model looks familiar at first (tables, rows, columns) and then behaves very differently from a relational table. If you carry over your SQL habits, you will design slow tables. This lesson names every part of the model precisely and shows how each part is stored.

You will meet namespaces, tables, rows, column families, column qualifiers, cells and values. Cells and versions get a lesson of their own next: Cells, Timestamps and Versions.

The hierarchy

From namespace to value

Namespace A group of tables, like a schema
Table Rows sorted by key
Row key Unique byte string, the only index
Column family Fixed at table creation

Qualifiers and cells live inside a family.

1 / 2
  • A namespace groups tables (like a database schema). Two namespaces exist by default: default and hbase (system tables). Table names are written namespace:table.
  • A table is a collection of rows.
  • A row is identified by a row key, an arbitrary byte array (commonly a string). Row keys are unique per table and the table is sorted by them.
  • A column family is a group of columns stored together. Families are part of the table's schema and must be declared when you create the table (you can alter them later, but rarely).
  • A column qualifier is the name of a column inside a family. Qualifiers are not declared: any put can invent a new one. A full column name is written family:qualifier, for example info:email.
  • A cell is the coordinate (row, family, qualifier, timestamp) plus the value.
  • Everything is bytes. HBase does not know types. Your client converts strings and numbers to bytes and back.

Why families matter so much

Each column family of each region is stored in its own set of files (its own Store). That has practical consequences:

  • Reading only info columns never touches the files of the photos family.
  • Each family has its own settings: compression, TTL, number of versions, block size, bloom filter type.
  • A flush or compaction handles a whole Store at a time. Many families means many small files and more work.

For these reasons the guidance is: use few families, ideally one or two, up to about three. Put columns that are read together in the same family, and put big or rarely-read data (such as raw binary blobs) in a separate family with its own settings. Use short family names (d, i) in large tables because the name is stored with every cell.

Sparse, sorted and ordered

Two properties explain most of HBase's behavior.

Sorted. Rows are ordered by row key, compared as unsigned bytes, one byte at a time. Within a row, cells are ordered by family, then qualifier, then timestamp (newest first). A scan reads in exactly this order.

Sparse. A row stores only the cells it has. There is no NULL on disk. A table with a million possible qualifiers costs nothing for qualifiers a row does not use. This makes wide, irregular rows practical, for example one qualifier per tag or per day.

The stand-in below builds a miniature table with a declared family set and free qualifiers. Try adding a family that was not declared: it is rejected, just as real HBase rejects it.

Java Stand-in: families are declared, qualifiers are free

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Row keys are bytes, compared lexicographically

This surprises everyone once. Row keys sort as bytes, so as strings "10" comes before "9", because '1' (0x31) is smaller than '9' (0x39). If you want numeric order, you must make the bytes sort that way, usually by zero-padding ("0009", "0010") or by using fixed-width binary integers.

TEXT
Sorted as text:   "1", "10", "100", "2", "20", "3"
Zero-padded:      "001", "002", "003", "010", "020", "100"

A related rule: a row key can be up to 64 KB, but you should keep it short (a few tens of bytes). The key is repeated in every cell, so a 100-byte key with ten cells wastes a kilobyte per row.

Java Stand-in: string order vs numeric order

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

What the real shell looks like

These real shell commands create a table with two families and write a value into a qualifier that was never declared. They are read-only here; the output of put is a timing line that varies, so it is omitted.

Terminal
create 'users', 'info', 'stats'
put 'users', 'u1', 'info:name', 'Jon'
put 'users', 'u1', 'info:twitter', '@jon'

Mistakes to avoid

  • Treating qualifiers like a schema you must register. You do not; but you must keep your own list of the names you use.
  • Too many families. Each family is a separate Store with its own memstore and files.
  • Long family and qualifier names on big tables: they are copied into every cell.
  • Assuming types. "42" and the 4-byte integer 42 are different byte arrays; reading with the wrong conversion yields garbage.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. When must a column family be defined?

    Families are fixed schema. Qualifiers, in contrast, can be invented by any put.
  2. 2. How are row keys ordered?

    HBase compares bytes, so "10" sorts before "9" unless you pad numbers.
  3. 3. Why is it recommended to keep the number of column families small?

    Flushes and compactions work per Store, and many small Stores multiply that work.
  4. 4. A row has no value for info:email. What is stored for it?

    Tables are sparse: missing cells cost no space.
  5. 5. Why use short family names like 'd' in huge tables?

    The name is part of every cell's key, so long names multiply storage.

Summary

  • Hierarchy: namespace, table, row key, column family, qualifier, cell, value (bytes).
  • Families are declared up front, kept few, and each has its own Store and settings.
  • Qualifiers are free-form and rows are sparse, so empty cells cost nothing.
  • Row keys sort as unsigned bytes; pad numbers and keep keys short.
  • HBase has no types: your code converts to and from bytes.

Next: Cells, Timestamps and Versions