Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 39 ยท Testing and Versioning

gRPC Load Testing and Performance Tuning

Measure gRPC performance with ghz and latency percentiles, then tune message size limits, compression, keepalive, connection use and streaming, with runnable Go measurements.

gRPC has a reputation for speed: binary protobuf messages are smaller than JSON, HTTP/2 multiplexes calls on one connection and codegen avoids reflection. But a reputation is not a measurement. Whether your service handles a thousand requests per second at a 50 ms 99th percentile is something you find out by testing, not by assuming.

This lesson covers how to measure (percentiles, load tools, honest test design) and the handful of knobs that actually matter when tuning.

The ghz tool and grpc-go cannot run on this site, so command lines and option code are read-only. The Go examples compute percentiles, compare payload sizes and demonstrate batching with the standard library, so the numbers you see are real measurements of those ideas.

Measure latency with percentiles, not averages

An average hides the pain. If 99 requests take 10 ms and one takes 5 seconds, the average is about 60 ms and sounds fine, but one user in a hundred waited five seconds. Report percentiles: p50 (the median), p90, p99 and p99.9. Since a page or workflow usually makes many calls, the chance that some call hits the slow tail is high, so the tail matters most.

Go Average versus percentiles

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The average looks harmless, while p99 reveals the slow path. Always read both, and track p99 in your metrics and alerts.

Load testing with ghz

ghz is a command-line load generator for gRPC. It reads your .proto (or uses reflection), fires calls and prints a latency histogram:

Terminal
ghz --insecure \
    --proto proto/users/v1/users.proto \
    --call users.v1.UserService.GetUser \
    -d '{"id": "42"}' \
    --connections 8 --concurrency 64 \
    --duration 30s --rps 2000 \
    localhost:50051

Key options: --connections (how many HTTP/2 connections), --concurrency (workers sending in parallel), --rps (a target rate, an open-loop load), --duration or -n (number of calls), -m to send metadata, --tls/--cacert for TLS. Other good tools are k6 (with its gRPC module) and Fortio.

Test design rules

  • Run the test from a separate machine with similar network distance to real clients; a load generator on the same host steals CPU from the server.
  • Use realistic payloads and data: tiny requests against a warm cache prove little.
  • Warm up first (JIT, connection pools, caches), then measure.
  • Beware coordinated omission: a closed-loop generator slows down when the server slows down, hiding the very delays you want to see. Prefer a fixed arrival rate (--rps) and record latency from the intended send time.
  • Change one thing at a time and repeat runs; report median of several.
  • Test failure modes too: kill a backend mid-test and watch errors and recovery.

Tuning knobs that matter

1. Message size limits

grpc-go servers reject received messages larger than 4 MB by default (ResourceExhausted). Raising the limit is easy, but a better question is why the message is that big:

Go
grpc.NewServer(grpc.MaxRecvMsgSize(16 << 20))      // server accepts up to 16 MB
grpc.NewClient(target, grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(16 << 20)))

For large data use streaming (many small messages) or send a reference to object storage.

2. Compression

gRPC supports per-message compression (gzip built in; zstd and snappy via extensions). It trades CPU for bytes: worthwhile for large, text-heavy messages over slow links, wasteful for small or already-compressed payloads (images, encrypted blobs).

Go
import _ "google.golang.org/grpc/encoding/gzip" // registers the compressor

client.Upload(ctx, req, grpc.UseCompressor(gzip.Name))
Go When does compression pay off?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Repetitive data shrinks dramatically; tiny and random payloads do not (the tiny message even grows, because gzip adds a header). Measure before enabling it globally.

3. Keepalive and connection health

Idle connections can be dropped silently by NAT gateways and load balancers. Keepalive pings detect dead peers early and keep paths open, but pinging too often triggers GOAWAY ... too_many_pings:

Go
// client
grpc.WithKeepaliveParams(keepalive.ClientParameters{
    Time:                30 * time.Second, // ping if idle this long
    Timeout:             10 * time.Second, // wait this long for the ack
    PermitWithoutStream: true,
})

// server must agree
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
    MinTime:             20 * time.Second,
    PermitWithoutStream: true,
})

Keep client Time at or above the server's MinTime. On the server, MaxConnectionAge makes connections recycle so that load balancing sees new backends.

4. Reuse connections and clients

A grpc.ClientConn is safe for concurrent use and multiplexes many calls. Create it once and share it; do not dial per request. A single connection typically allows about 100 concurrent streams (the server-advertised MAX_CONCURRENT_STREAMS); a very busy client may need a small pool of connections, which is what --connections in ghz explores.

5. Batch or stream instead of chatty calls

Fifty tiny calls cost fifty round trips and fifty sets of headers. One call carrying fifty items costs one. This is usually the single biggest win:

Go Chatty calls versus one batch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Design APIs with BatchGet... methods, or let callers issue calls in parallel on one connection, or use a bidirectional stream when messages flow continuously.

6. Server-side costs

Often the bottleneck is not gRPC but what the handler does: database queries, locks, allocations. Profile with pprof, watch goroutine counts, size connection pools to the database and make sure handlers respect deadlines so abandoned work stops. Reuse protobuf messages carefully (avoid sharing mutable messages between goroutines) and avoid converting to JSON and back inside hot paths.

A tuning checklist

  1. Measure first; record p50/p99, throughput, CPU and memory.
  2. Fix chatty APIs and N+1 call patterns.
  3. Check message sizes; stream big payloads.
  4. Enable compression only where measurements show a win.
  5. Configure keepalive consistently on both sides.
  6. Share one ClientConn per target; add a small pool only if you hit stream limits.
  7. Re-measure after each change.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why report p99 latency in addition to the average?

    Averages hide slow outliers. Percentiles show what the slowest calls really cost.
  2. 2. What is the default maximum received message size in grpc-go servers?

    The default is 4 MB; larger messages fail with RESOURCE_EXHAUSTED unless you raise the limit or stream the data.
  3. 3. When is compression least useful?

    Small messages gain little and compressed or random data does not shrink, so you only spend CPU.
  4. 4. What is the typical biggest win for chatty clients?

    Fewer round trips beat almost any micro-optimization.
  5. 5. A client keepalive Time of 5 seconds with a server MinTime of 20 seconds will most likely cause:

    The server enforces a minimum ping interval and closes connections that ping more often.

Summary

  • Measure with percentiles, from a separate machine, with realistic payloads, warm-up and a fixed arrival rate.
  • ghz generates gRPC load from your protos; k6 and Fortio are alternatives.
  • The knobs that matter: message size limits, selective compression, consistent keepalive, shared connections, batching or streaming.
  • Server-side work (database, locks) is usually the real bottleneck; profile it.
  • Change one thing at a time and re-measure.

Next: Project: A Key-Value Store Service