Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 39 ยท Load Balancing and Gateways

REST Transcoding with gRPC-Gateway

Serve REST and JSON from a gRPC service with google.api.http annotations, grpc-gateway and Envoy transcoding, with a runnable Go path-template router and JSON mapping demo.

gRPC is wonderful between services, but not every client speaks it. Browsers cannot make arbitrary gRPC calls, partners want plain curl examples, and many tools only understand JSON over HTTP. You do not want to write and maintain two APIs. The answer is transcoding: describe how each RPC maps to an HTTP route in the .proto file, and let a gateway translate between REST/JSON and gRPC.

Two popular implementations: grpc-gateway, a Go code generator that creates a reverse proxy server, and Envoy's gRPC-JSON transcoder filter, which does the same inside your proxy using a compiled descriptor. Both read the same annotations.

The generators and the gRPC runtime are not available on this site, so annotations and generated-code usage are shown read-only. The runnable blocks implement the core mapping logic (path templates, body and query mapping, JSON naming, status to HTTP) in plain Go, to show exactly what a gateway does for every request.

Annotating the service

The annotation lives in google/api/http.proto, imported into your file:

PROTO
syntax = "proto3";
package users.v1;

import "google/api/annotations.proto";

service UserService {
  rpc GetUser(GetUserRequest) returns (User) {
    option (google.api.http) = {
      get: "/v1/users/{id}"
    };
  }

  rpc ListUsers(ListUsersRequest) returns (ListUsersResponse) {
    option (google.api.http) = {
      get: "/v1/users"
    };
  }

  rpc CreateUser(CreateUserRequest) returns (User) {
    option (google.api.http) = {
      post: "/v1/users"
      body: "user"
    };
  }

  rpc UpdateUser(UpdateUserRequest) returns (User) {
    option (google.api.http) = {
      patch: "/v1/users/{user.id}"
      body: "user"
    };
  }

  rpc DeleteUser(DeleteUserRequest) returns (google.protobuf.Empty) {
    option (google.api.http) = { delete: "/v1/users/{id}" };
  }
}

The mapping rules are simple and worth memorizing:

  1. in the path fills the request field id. A dotted name like {user.id} fills a nested field.
  2. body: "user" means the JSON body is the user field of the request message. body: "*" means the body is the whole request message.
  3. Any request field that is not in the path or the body becomes a query parameter (/v1/users?page_size=20&filter=active).
  4. For GET and DELETE there is normally no body.
  5. additional_bindings lets one RPC have several routes.

The JSON mapping

Protobuf has a canonical JSON mapping, so gateways agree with each other: field names become lowerCamelCase (page_size to pageSize, though parsers accept both), int64 values are strings (to avoid losing precision in JavaScript), enums are their names, bytes are base64, Timestamp is an RFC 3339 string and oneof fields appear as whichever member is set. Default values are omitted unless you ask to emit them.

Using grpc-gateway (real API)

Terminal
# buf.gen.yaml has plugins: go, go-grpc, grpc-gateway, openapiv2
buf generate

The generator creates users.pb.gw.go. You then run the gateway in front of your gRPC server:

Go
mux := runtime.NewServeMux()
opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
if err := pb.RegisterUserServiceHandlerFromEndpoint(ctx, mux, "localhost:50051", opts); err != nil {
    log.Fatal(err)
}
log.Fatal(http.ListenAndServe(":8080", mux))
Terminal
curl http://localhost:8080/v1/users/42
curl -X POST http://localhost:8080/v1/users \
     -H 'content-type: application/json' \
     -d '{"name": "Ada", "email": "[email protected]"}'

The same buf generate can produce an OpenAPI (Swagger) document from the annotations, so your REST documentation always matches the real routes. Metadata passes through: HTTP headers like Authorization become gRPC metadata (a configurable allowlist, plus the Grpc-Metadata- prefix), and gRPC status codes become HTTP statuses.

Mapping gRPC errors to HTTP

The gateway converts status codes to HTTP statuses and renders the google.rpc.Status as JSON, including rich details:

JSON
{
  "code": 3,
  "message": "invalid user",
  "details": [
    { "@type": "type.googleapis.com/google.rpc.BadRequest",
      "fieldViolations": [{ "field": "email", "description": "must contain @" }] }
  ]
}

Runnable: a path template router

This program compiles templates like /v1/users/{id} and matches request paths, then builds a request by combining path variables, body and query parameters, which is the heart of a gateway.

Go Match HTTP routes to RPC requests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Runnable: canonical JSON naming

A gateway must also rename fields and treat 64-bit integers specially. This demo shows the two rules that surprise people most:

Go Protobuf JSON: camelCase and int64 as strings

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The last line shows why int64 becomes a string: 9007199254740993 does not fit exactly in a double, so JavaScript would silently change it.

Design tips

  • Design the REST surface on purpose: follow resource-oriented patterns (/v1/users/{id}, standard methods Get, List, Create, Update, Delete) and make the gRPC API fit them from the start.
  • Version in the path (/v1) and in the proto package (users.v1); see schema evolution.
  • Streaming RPCs are limited: server streams become newline-delimited JSON, client and bidirectional streams are not supported by REST. Keep a native gRPC route for them.
  • Run the gateway as a separate process or sidecar so REST clients cannot reach the raw gRPC port, and apply authentication at the gateway as well.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In an http annotation, what happens to request fields that are in neither the path nor the body?

    Remaining fields are read from the URL query string, for example ?page_size=20.
  2. 2. What does body: "*" mean?

    With * the entire request message is read from the body; a field name maps the body to just that field.
  3. 3. Why are int64 values strings in canonical protobuf JSON?

    Doubles lose precision above 2^53, so protobuf JSON uses strings for 64-bit integers.
  4. 4. Which kind of RPC cannot be fully exposed through plain REST transcoding?

    REST has no equivalent for a bidirectional stream, so keep a native gRPC (or gRPC-Web/WebSocket) route for those.

Summary

  • google.api.http annotations map RPCs to HTTP routes: path variables, body, and query parameters for the rest.
  • grpc-gateway generates a Go reverse proxy; Envoy's transcoder does the same inside the proxy; both can generate OpenAPI.
  • Protobuf JSON uses lowerCamelCase names and strings for int64.
  • gRPC status codes map to HTTP statuses and error details render as JSON.
  • Design REST routes deliberately, version them, and keep streaming on native gRPC.

Next: gRPC-Web and Proxies