Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 36 ยท Shell and Basic Operations

Shell: Scan and Count

Learn the HBase scan command: row ranges, column selection, limits, reversed scans, prefix and value filters, raw scans for tombstones, and how to count rows efficiently.

get reads one row you already know. scan reads many rows in key order, and it is the operation that sorted storage makes cheap. A scan starts at a row, walks forward through the sorted table, and stops at a row, a limit or the end. This lesson shows every option you will use in the shell. The commands are read-only listings; a Java stand-in lets you run the same logic.

Examples use this table:

Terminal
create 'logs', {NAME => 'd', VERSIONS => 1}
put 'logs', 'web01#2025-01-01', 'd:status', '200'
put 'logs', 'web01#2025-01-02', 'd:status', '500'
put 'logs', 'web02#2025-01-01', 'd:status', '200'
put 'logs', 'web02#2025-01-03', 'd:status', '404'

The basic scan

Terminal
scan 'logs'

This reads every cell of every row, in row key order. On a big table it will run for a long time, so always limit scans on real data.

Terminal
scan 'logs', {LIMIT => 10}

The output lists ROW and COLUMN+CELL pairs, then a summary line with the row count. One output line represents one cell, not one row.

Ranges: STARTROW and STOPROW

Terminal
scan 'logs', {STARTROW => 'web01#', STOPROW => 'web02#'}

STARTROW is inclusive, STOPROW is exclusive. This is the most important scan idiom, because it uses the sorted order to read only one contiguous slice. To get all rows beginning with a prefix, the stop row is the prefix with its last character incremented: for the prefix web01# that is web01$, because $ (0x24) comes right after # (0x23). The shell also has a prefix filter (below) which does the same job.

Terminal
scan 'logs', {ROWPREFIXFILTER => 'web01#'}

ROWPREFIXFILTER sets suitable start and stop rows for you, so the scan does not read the rest of the table.

Choosing columns and versions

Terminal
scan 'logs', {COLUMNS => ['d:status']}               # only some columns
scan 'logs', {COLUMNS => 'd'}                        # a whole family
scan 'logs', {COLUMNS => ['d:status'], VERSIONS => 3}
scan 'logs', {TIMERANGE => [1735689600000, 1735776000000]}

Selecting columns is a real optimization: families you do not ask for are not read at all, because each family has its own files.

Reversed scans

Terminal
scan 'logs', {REVERSED => true, LIMIT => 3}

A reversed scan starts from the end (or from STARTROW going backwards) and yields rows in descending order. It is slower than a forward scan, but ideal for "latest N" when time is the last part of the key.

Filters

Filters run on the server and drop cells or rows before they cross the network.

Terminal
scan 'logs', {FILTER => "PrefixFilter('web01')"}
scan 'logs', {FILTER => "ValueFilter(=, 'binary:200')"}
scan 'logs', {FILTER => "SingleColumnValueFilter('d', 'status', =, 'binary:500')"}
scan 'logs', {FILTER => "PageFilter(5)"}
scan 'logs', {FILTER => "KeyOnlyFilter()"}
scan 'logs', {FILTER => "FirstKeyOnlyFilter() AND KeyOnlyFilter()"}

Important: a filter does not make a scan faster by itself. HBase still reads the rows between your start and stop row; the filter only reduces what is sent back. Narrow the range first, then filter. The Filters and Coprocessors chapter of the HBase track covers filters in depth.

Raw scans and tombstones

Terminal
scan 'logs', {RAW => true, VERSIONS => 10}

A raw scan returns everything the files hold, including delete markers and cells hidden by them. It is the tool for investigating "I deleted it but it comes back" (see Cells, Timestamps and Versions).

Scan caching

The client does not fetch one row per network call. CACHE controls how many rows a call to the server returns; BATCH limits cells per row per call (for very wide rows); MAXRESULTSIZE caps bytes:

Terminal
scan 'logs', {CACHE => 500, LIMIT => 2000}

Bigger caches mean fewer round trips but more client memory and a longer time between calls, which can hit the scanner lease timeout (60 seconds by default, hbase.client.scanner.timeout.period).

Count

Terminal
count 'logs'
count 'logs', INTERVAL => 100000, CACHE => 1000

count is a scan under the hood: it reads every row, so it is slow on large tables. It prints a progress line every INTERVAL rows. For a quick estimate in a big table use the row counter MapReduce job (hbase org.apache.hadoop.hbase.mapreduce.RowCounter 'logs') or table metrics, not a shell count.

A scan in Java

This stand-in implements start/stop rows, limit, column choice, reversed scans and prefix filtering over a sorted map, so you can see why a range scan touches only a slice of the data.

Java Stand-in: scan with range, limit and reverse

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Look at the examined numbers. The range scan examined only the rows inside the range; the prefix filter over the whole table examined all five rows even though it returned two. That is the lesson: ranges save work, filters only save bandwidth.

Counting rows with a cursor

Counting without loading whole rows is possible when the server sends only the first cell of each row. FirstKeyOnlyFilter plus KeyOnlyFilter do that in real HBase. The second stand-in counts rows per server prefix, the way a "rows per host" report would.

Java Stand-in: counting rows per prefix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In scan 'logs', {STARTROW => 'a', STOPROW => 'b'}, which bound is included?

    STARTROW is inclusive and STOPROW is exclusive.
  2. 2. Why is a server-side filter without a row range often still slow?

    Filters reduce network traffic, not the rows examined. Start and stop rows reduce the work.
  3. 3. What is a RAW scan useful for?

    Raw scans return tombstones and masked data, which helps debug deletes.
  4. 4. Why is the shell count command slow on a very large table?

    count is implemented as a scan; use RowCounter or metrics for big tables.

Summary

  • scan walks rows in key order; always add LIMIT or a range on real data.
  • STARTROW is inclusive, STOPROW exclusive; ROWPREFIXFILTER sets both for a prefix.
  • COLUMNS limits families read; REVERSED scans backwards; RAW shows tombstones.
  • Filters cut the data returned, not the rows examined; narrow with a range first.
  • count scans everything: use RowCounter or metrics for large tables.

Next: Shell: Namespaces, TTL and Snapshots