Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 36 ยท Java API

Java Client: Scans, Batches and BufferedMutator

Scan HBase tables from Java with ranges, caching and filters, close ResultScanners correctly, send mixed batches with Table.batch, and buffer heavy writes with BufferedMutator.

Single get and put calls are fine for a web request, but real programs also process ranges of rows and load data in bulk. This lesson covers three tools: the Scan for reading ranges, Table.batch for sending many different operations in one go, and BufferedMutator for high-volume writes. Real API code is read-only (no cluster is available); Java stand-ins reproduce the mechanics you can run.

Scan

Java
Scan scan = new Scan()
    .withStartRow(Bytes.toBytes("web01#"))          // inclusive
    .withStopRow(Bytes.toBytes("web01$"))           // exclusive
    .addFamily(Bytes.toBytes("d"))
    .setLimit(1000)                                  // stop after 1000 rows
    .setCaching(200);                                // rows fetched per RPC

try (ResultScanner scanner = table.getScanner(scan)) {
    for (Result row : scanner) {
        String key = Bytes.toString(row.getRow());
        byte[] status = row.getValue(Bytes.toBytes("d"), Bytes.toBytes("status"));
        // process
    }
}

Key points:

  • withStartRow/withStopRow replace the old setStartRow/setStopRow in 2.x. withStartRow(row, false) makes the start exclusive; withStopRow(row, true) makes the stop inclusive. setRowPrefixFilter(prefix) builds a range for a prefix.
  • Always close the ResultScanner. It holds a server-side scanner and a lease; leaking it wastes server memory.
  • setCaching(n) controls rows per RPC (the default is large, often Integer.MAX_VALUE limited by hbase.client.scanner.max.result.size; check your version). Large values mean fewer round trips but more client memory and a longer gap between calls.
  • setBatch(n) limits cells per Result for very wide rows, so one row may arrive as several Result objects.
  • setMaxResultSize(bytes) bounds the bytes per RPC.
  • setReversed(true) scans backwards; the start row is then the highest row to examine.
  • setCacheBlocks(false) keeps a large scan out of the block cache (see The Read Path).
  • setFilter(Filter) pushes a predicate to the server. Narrow the row range first; a filter alone does not reduce the rows examined.
  • setTimeRange / readVersions work as in Get.
  • setAllowPartialResults(true) lets huge rows be returned in pieces.

A scanner survives only while the client keeps asking: if your loop does slow work between next() calls for longer than hbase.client.scanner.timeout.period (60 s default), the server drops the scanner and you get ScannerTimeoutException or UnknownScannerException. Lower the caching or process rows faster.

Prefix scans and the "increment the last byte" trick

To scan all keys starting with a prefix without setRowPrefixFilter, the stop row is the prefix with its last byte incremented. The following stand-in computes it with byte arithmetic, including the case where the last byte is 0xFF (it must be dropped and the previous byte incremented).

Java Stand-in: the stop row for a prefix scan

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Scan caching, simulated

The next stand-in counts "RPC calls" for a scan of 1,000 rows with different caching values. Fewer calls means less latency overhead, but each call carries more data.

Java Stand-in: caching size vs number of RPCs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Table.batch: many operations, one call

table.put(List<Put>) sends puts only. To mix gets, puts, deletes and increments, build a list of Row objects and call batch:

Java
List<Row> actions = new ArrayList<>();
actions.add(new Put(Bytes.toBytes("u001")).addColumn(CF, Bytes.toBytes("name"), Bytes.toBytes("Jon")));
actions.add(new Get(Bytes.toBytes("u002")));
actions.add(new Delete(Bytes.toBytes("u003")));

Object[] results = new Object[actions.size()];
try {
    table.batch(actions, results);
} catch (RetriesExhaustedWithDetailsException e) {
    // some actions failed: inspect e.getNumExceptions(), e.getRow(i), e.getCause(i)
}

The client groups actions by RegionServer and sends them in parallel. Important facts: there is no ordering guarantee between actions (do not put and get the same row in one batch and expect order); actions are not atomic as a group (each is atomic on its own row); and the results array tells you what happened for each action: a Result for gets, an empty Result for successful mutations, or the exception object for a failed one.

The stand-in sends a mixed batch, collects per-action results and shows partial failure handling.

Java Stand-in: a mixed batch with partial failure

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

BufferedMutator: fast, asynchronous writes

Sending one put per call is slow for bulk loads because each call waits for the server. BufferedMutator collects mutations in a client-side buffer and sends them in large batches in the background:

Java
BufferedMutator.ExceptionListener listener = (e, mutator) -> {
    for (int i = 0; i < e.getNumExceptions(); i++) {
        System.err.println("Failed: " + e.getRow(i));
    }
};
BufferedMutatorParams params = new BufferedMutatorParams(TableName.valueOf("events"))
    .writeBufferSize(8 * 1024 * 1024)       // flush when 8 MB are buffered
    .listener(listener);

try (BufferedMutator mutator = connection.getBufferedMutator(params)) {
    for (Event event : events) {
        mutator.mutate(new Put(Bytes.toBytes(event.key()))
            .addColumn(CF, Bytes.toBytes("v"), Bytes.toBytes(event.payload())));
    }
    mutator.flush();      // force out whatever is left; close() also flushes
}

Things to know:

  • mutate returns quickly; data is not durable until the buffer is flushed. If the process dies, buffered edits are lost.
  • Errors surface later through the listener (or an exception from a later mutate/flush), not at the call that caused them.
  • BufferedMutator is thread-safe: one instance can be shared by many threads (unlike Table).
  • It accepts Put, Delete, Increment and Append.
  • Pick a buffer size between a few MB and tens of MB; very large buffers delay failures and use memory.

Below, a stand-in buffer that flushes at a size threshold, with a count of "RPCs" saved compared with one call per mutation.

Java Stand-in: a write buffer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Choosing between them

Need Use
Read a range of rows Scan
Read several known rows table.get(List<Get>)
Mixed operations in one call table.batch
Many puts with best throughput BufferedMutator (or table.put(List<Put>) for modest sizes)
Huge one-time load Bulk load of HFiles (see the write path lesson)

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What must you always do with a ResultScanner?

    A scanner keeps server-side state; leaking it wastes resources and can exhaust the server.
  2. 2. What does a larger setCaching value do?

    Caching is the number of rows fetched per next() RPC.
  3. 3. Which is true for Table.batch?

    Each action is atomic on its own row only; the results array shows per-action outcomes.
  4. 4. What is the main risk of BufferedMutator?

    Mutations sit in a client buffer until flushed, so durability and error reporting are delayed.
  5. 5. Why can a slow scan loop fail with ScannerTimeoutException?

    The scanner lease (60 seconds by default) expires when the client is too slow between calls.

Summary

  • Scan with withStartRow/withStopRow reads a key range; set setLimit, setCaching and setCacheBlocks(false) deliberately; always close the scanner.
  • Prefix scans use a stop row computed by incrementing the last non-0xFF byte.
  • Table.batch runs mixed actions in parallel per server, with per-action results and no ordering guarantee.
  • BufferedMutator buffers writes for throughput; flush before exit and handle errors through the listener.
  • Choose the tool by the access pattern: scan, multi-get, batch, buffered writes, or bulk load.

Next: Java Client: The Admin API