Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 36 ยท Integration and Patterns

Thrift, REST and Apache Phoenix

Access HBase from any language with the REST and Thrift gateways, or use SQL through Apache Phoenix: endpoints, encoding, secondary indexes and trade-offs.

The native HBase client is Java. Applications in Python, PHP, Go or JavaScript need another door in. HBase ships two gateway servers, REST and Thrift, that speak a language-neutral protocol and translate it into native client calls. And if your team wants SQL, Apache Phoenix layers a SQL engine, a type system and secondary indexes on top of HBase.

HBase and Phoenix cannot run in the live editors on this site. Requests, responses and SQL are shown as read-only text. The runnable Java blocks are models of the encoding details (Base64 JSON of the REST gateway and the composite key encoding Phoenix uses), labelled as such.

The REST gateway

Start it with hbase rest start (default port 8080, so check conflicts with other services). It maps URLs to tables and rows:

Request Meaning
GET / List tables
GET /events/schema Table schema
GET /events/row1 Read a row
GET /events/row1/d:status Read a column
PUT /events/row1 Write a row
DELETE /events/row1 Delete a row
PUT /events/scanner Create a scanner (then GET the returned URL)

With Accept: application/json row keys, column names and values come back Base64 encoded, because HBase keys and values are arbitrary bytes:

TEXT
GET /events/row1
Accept: application/json

{"Row":[{"key":"cm93MQ==","Cell":[{"column":"ZDpzdGF0dXM=","timestamp":1731400000000,"$":"b2s="}]}]}

cm93MQ== is row1, ZDpzdGF0dXM= is d:status, and b2s= is ok. A write has the same shape. Content types supported include JSON, XML and protobuf (the most compact).

Java Base64 in the REST JSON (model)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The last line is the usual surprise: a value written as a binary long through the native client appears as AAAAAAAAA44=, not as 910. Agree on one encoding across all clients (text or fixed-width binary) before mixing REST and Java.

REST is easy to try with curl, but each call has HTTP overhead and no batching beyond what you send in one body, so it suits light traffic, tools and prototypes.

The Thrift gateway

Apache Thrift generates client code for many languages from an interface file. Start the server with hbase thrift start (Thrift 2 API, the recommended one, on port 9090; thrift 1 is the legacy API). Libraries such as Python's HappyBase use it:

Python
# read-only: Python client talking to the Thrift server
import happybase
conn = happybase.Connection('hbase-thrift-host', port=9090)
table = conn.table('events')
table.put(b'row1', {b'd:status': b'ok', b'd:ms': b'12'})
row = table.row(b'row1')
for key, data in table.scan(row_prefix=b'user42|', limit=10):
    print(key, data)

Thrift is binary and persistent, so it is faster than REST, and it has proper scanner and batch calls. The gateway is stateless except for open scanners: run several behind a load balancer, and make scan sessions stick to one instance. Both gateways are extra hops: they add latency and cap throughput, and they must be secured (SSL, Kerberos, doAs impersonation) like the cluster itself.

Apache Phoenix: SQL on HBase

Phoenix is a SQL layer that compiles queries into HBase scans and pushes work into the region servers with coprocessors. You get typed columns, composite primary keys, joins, aggregation, UPSERT, secondary indexes and views. Applications connect with a JDBC driver (or a thin query server for other languages).

SQL
CREATE TABLE events (
  user_id  VARCHAR NOT NULL,
  ts       TIMESTAMP NOT NULL,
  status   VARCHAR,
  ms       BIGINT,
  CONSTRAINT pk PRIMARY KEY (user_id, ts)
) SALT_BUCKETS = 8, COMPRESSION = 'SNAPPY';

UPSERT INTO events VALUES ('user42', NOW(), 'error', 910);

SELECT status, COUNT(*), AVG(ms)
FROM events
WHERE user_id = 'user42' AND ts > CURRENT_DATE() - 7
GROUP BY status;

CREATE INDEX idx_status ON events (status) INCLUDE (ms);

Key facts that connect to what you learned earlier:

  • The primary key is the HBase row key: Phoenix concatenates user_id and ts into a composite key with proper encodings (see row key design), so a WHERE user_id = ... is a prefix scan, while a query on status alone is a full scan unless an index exists.
  • SALT_BUCKETS adds a salt byte and pre-splits the table automatically: salting from hotspotting, done for you; queries fan out to all buckets.
  • Secondary indexes are separate HBase tables maintained by coprocessors: global indexes (cheap reads, extra write cost), local indexes (stored in the same region) and covered indexes (INCLUDE columns so the index answers the query without going back to the main table). These automate the index-table pattern from schema patterns.
  • You can map an existing HBase table read-only with CREATE VIEW, but columns written by non-Phoenix clients must follow Phoenix's byte encodings.
Java Composite key encoding like Phoenix (model)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Notice that the negative timestamp -5 sorts before the positive ones, thanks to the flipped sign bit, and the user42 prefix scan is a short range.

Choosing an access method

Option Best for Watch out for
Native Java client Services in the JVM, highest throughput Java only
REST gateway Quick scripts, tooling, low traffic Base64 payloads, HTTP overhead
Thrift gateway Python, PHP and others at moderate load Gateway sizing, scanner stickiness
Phoenix SQL, typed columns, secondary indexes, BI tools Extra layer to run; data encodings must match

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does the REST gateway return Base64 strings in JSON?

    HBase stores raw bytes, so a text-only format like JSON needs an encoding such as Base64.
  2. 2. Which gateway offers binary protocol clients in many languages?

    Thrift generates clients for many languages and uses a compact binary protocol.
  3. 3. In Phoenix, what is the HBase row key?

    Phoenix encodes the composite primary key into the row key, so key design rules still apply.
  4. 4. What does SALT_BUCKETS do in Phoenix?

    It is automatic salting plus pre-splitting; queries then fan out over the buckets.

Summary

  • REST and Thrift gateways let non-Java apps use HBase; Thrift is faster, REST is simpler.
  • Gateways are extra hops: size them and secure them.
  • Phoenix adds SQL, types and secondary indexes on top of HBase row keys.
  • Agree on one value encoding across all clients.

Next: Bulk Loading HFiles