Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 36 ยท Filters and Coprocessors

Filters and Scan Basics

Learn how HBase filters work on the server side: row, column and value filters, scan ranges, paging and what filters cannot do, with Java stand-ins.

A Scan reads a range of rows. A filter is a small piece of logic that runs on the region server while it scans and decides which rows, columns or cells to return. Filtering on the server saves network traffic: instead of shipping a million rows to the client to throw most of them away, the region server drops them first.

Filters are powerful but easy to misuse. The most important thing to remember is that a filter does not make a scan fast by itself. A filter still reads the data it examines; it only reduces what is sent back. The fastest "filter" is a narrow scan range. This lesson explains both.

Real HBase is not available in the live editors here. HBase client calls are shown as read-only code. The runnable blocks are Java stand-ins that apply the same ideas to a sorted in-memory table so you can see what gets returned.

Narrow the scan first

A scan has a start row (inclusive) and a stop row (exclusive). Both use the sort order of the row keys, so a key prefix can be turned into a range. Always prefer a range to a filter when you can.

Java
// read-only: real HBase 2.x client
Scan scan = new Scan()
    .withStartRow(Bytes.toBytes("user42|"))
    .withStopRow(Bytes.toBytes("user42|๏ฟฟ"))   // or the prefix incremented by one byte
    .addFamily(Bytes.toBytes("d"))                  // only family d
    .setLimit(100)                                  // stop after 100 rows
    .setCaching(100);                               // rows per RPC round trip

Other settings that matter: addColumn limits which columns are read; setBatch splits very wide rows into chunks of N cells; setReversed(true) scans backwards; setTimeRange limits cell timestamps; setCacheBlocks(false) stops a one-off big scan from evicting hot blocks from the block cache.

The main filters

Filter Question it answers
PrefixFilter Row key starts with this prefix
RowFilter Row key matches a comparison (equal, less, regex, substring)
FamilyFilter, QualifierFilter Which families or column qualifiers to keep
ColumnPrefixFilter, ColumnRangeFilter Qualifiers by prefix or range
ValueFilter A cell's value matches
SingleColumnValueFilter The row qualifies when one specific column has a value
PageFilter At most N rows (approximate across regions)
KeyOnlyFilter, FirstKeyOnlyFilter Return keys only, or only the first cell of each row (cheap row counting)
TimestampsFilter Only cells with exactly these timestamps

Comparison filters take a comparator: BinaryComparator (exact bytes), BinaryPrefixComparator, SubstringComparator, RegexStringComparator, LongComparator. Mixing them is how you express "value equals 5" or "value contains error".

Filters see data in key order and they cost reads

A filter such as ValueFilter must read every cell in the scanned range to decide. If you scan a billion rows with a value filter, HBase reads a billion rows server-side and returns few. That is acceptable for an offline job and awful for a user-facing request. Rule of thumb: key range first, filters second.

Some filters are smart. PrefixFilter alone still starts scanning from the first row of the table unless you also set the start row, so combine it with withStartRow(prefix). FirstKeyOnlyFilter lets a row counter skip the other cells of each row.

A stand-in scan with filters

The program below models a table as a sorted map of rows to columns and implements a few filters as plain Java predicates. The point is not the data structure: it is seeing which rows each combination returns.

Java Scan ranges and filters (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

All four scans can return the same user-42 rows, but they examine very different numbers of rows: the prefix filter on a full scan looks at every row, the range scans look only at user 42's rows, and the limit stops early.

Paging through results

A user interface wants "page 2". HBase has no offset. The pattern is to remember the last row key you returned and start the next scan just after it: set the start row to lastKey + 0x00 (the smallest key greater than lastKey) and keep the same stop row. PageFilter is only a hint per region server, so rely on setLimit and your own counting for exact pages.

Java
// read-only
byte[] next = Bytes.add(lastRowKey, new byte[] {0});
Scan page2 = new Scan().withStartRow(next, true).setLimit(20);

Null columns and missing values

SingleColumnValueFilter has a trap. By default, rows that do not have the tested column are included in the results. Call setFilterIfMissing(true) to drop them. It also looks at the newest version by default; use setLatestVersionOnly(false) to test older versions.

Java
SingleColumnValueFilter f = new SingleColumnValueFilter(
    Bytes.toBytes("d"), Bytes.toBytes("status"),
    CompareOperator.EQUAL, Bytes.toBytes("error"));
f.setFilterIfMissing(true);     // skip rows with no d:status at all
scan.setFilter(f);
Java The missing-column trap (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

With the default setting, the row r3, which never had a status, comes back with the errors. Setting filterIfMissing removes it.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the main benefit of a server-side filter?

    The region server still examines the data, but only matching results cross the network.
  2. 2. How do you make a scan for rows starting with a prefix efficient?

    A key range lets HBase skip all other regions and rows; PrefixFilter alone would still start at the table's first row.
  3. 3. By default, what happens to rows that lack the column tested by SingleColumnValueFilter?

    Rows missing the column pass unless you call setFilterIfMissing(true).
  4. 4. How do you fetch the next page after the last returned row key?

    HBase has no offset; the next scan starts at the smallest key greater than the last one returned.

Summary

  • A scan range is the cheapest filter; combine it with server-side filters for the rest.
  • Filters reduce network traffic, not the rows examined.
  • Know the main filters and comparators; watch the missing-column default.
  • Page by remembering the last key; HBase has no offset.

Next: Filter Lists and Custom Filters