Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 39 ยท Performance and Scaling

Batching and Write Performance

Speed up InfluxDB writes with batching, sensible batch sizes, compression, timestamp ordering and retry with backoff. Runnable simulations of the trade-offs.

Writing one point per HTTP request is the most common way to make InfluxDB look slow. Each request pays for a network round trip, authentication, parsing and a commit. Sending a thousand points in one request pays those costs once. This lesson covers how to batch, how big a batch should be, what to do when a write fails, and a handful of settings that cut load.

The server is not running in this course, so the real client code is read-only, and the numbers are produced by runnable simulations that you can adjust.

Why batching works

Think of a delivery van. Taking one parcel per trip is wasteful, and the fixed cost of each trip dominates. The same holds for writes: a request has a fixed overhead (connection, headers, auth check, write-ahead log commit) plus a per-point cost. Total time is roughly requests x overhead + points x cost. Batching reduces the first term.

JS Stand-in: cost of sending 100,000 points

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The numbers are made up, but the shape is real: the first few steps of batching help enormously, then the gain flattens. Beyond a few thousand points per request, a bigger batch gives little speed-up and makes failures costlier.

How big should a batch be?

Common guidance from the InfluxDB documentation is in the range of 5,000 to 10,000 lines per request, or a few megabytes at most. Tune with your own data:

  • Too small and request overhead dominates.
  • Too large and a single failed request must be retried in full, memory use spikes, and the server may reject the body for size.
  • Add a flush interval (for example 1 second) so that a quiet period does not leave points sitting in the buffer.

Client libraries implement this for you with a batching write mode: points are queued and flushed when either the batch size or the flush interval is reached. A read-only Python example (InfluxDB 2.x client):

Python
from influxdb_client import InfluxDBClient, Point, WriteOptions

with InfluxDBClient(url="http://localhost:8086", token=TOKEN, org="my-org") as client:
    with client.write_api(write_options=WriteOptions(
        batch_size=5000, flush_interval=1000,
        jitter_interval=200, retry_interval=5000, max_retries=5,
        exponential_base=2,
    )) as writer:
        for i in range(100_000):
            writer.write(bucket="raw", record=Point("cpu").tag("host", "web-1").field("usage", 41.5))

The with block flushes on exit, so points are not lost when your program ends. Forgetting to flush or close is a classic cause of "missing" data.

Retries with exponential backoff and jitter

Writes fail: a timeout, a 429 "too many requests", a 503 while the server restarts. Retrying immediately makes overload worse. The standard answer is exponential backoff: wait 1 s, then 2 s, then 4 s, and so on up to a cap, plus random jitter so that thousands of clients do not retry at the same instant.

Which errors are worth retrying?

Response Retry?
Timeout, connection error, 5xx, 429 Yes, with backoff (honour a Retry-After header)
400 (malformed line protocol, type conflict) No: the data is wrong, fix it
401, 403 (bad token or permission) No: fix the credentials
404 (bucket or database missing) No
413 (payload too large) Split the batch, then retry

This simulation computes a retry schedule and shows jitter spreading clients apart.

JS Stand-in: exponential backoff schedule with jitter

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Each client waits about the same overall, but at different moments. That prevents a "thundering herd" when a server returns.

Compression, precision and ordering

A few settings that reduce work without changing your data:

  1. Gzip the request body. Line protocol is repetitive text and compresses very well. Send Content-Encoding: gzip. Client libraries expose a "gzip" option.
  2. Use the coarsest timestamp precision you can. The write API accepts precision (ns, us, ms, s). Writing second-resolution data as seconds avoids sending nine-digit nanosecond suffixes.
  3. Send points in time order, or at least close to it. Data arriving in order for the recent time range is cheapest for the storage engine; heavy back-filling of old data causes compactions across many files.
  4. Sort tags by key in line protocol. The server sorts them anyway, so pre-sorting saves it work and keeps series keys canonical.
  5. Keep a persistent connection (HTTP keep-alive) and reuse the client. Creating a client per write wastes the handshake.
  6. Write in parallel from several workers, each sending its own batches, once one connection is not enough. Watch the server's CPU and the 429 rate.

The next simulation measures the payload size effect of precision on the same points.

JS Stand-in: payload size of ns versus s timestamps

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Backpressure: what if the server is slower than you?

If your producer is faster than InfluxDB can accept, buffers grow until the process runs out of memory. Decide what happens: block the producer, drop the oldest points, or spill to disk. Telegraf has a buffer limit (metric_buffer_limit) and drops the oldest metrics beyond it. Good clients cap their queue and expose a counter for dropped points, which you should graph and alert on.

Common mistakes

  • One write call per point.
  • Not closing or flushing the writer at program exit.
  • Retrying 400 errors forever.
  • Retrying without backoff and jitter.
  • Back-filling a year of data in one burst through the same path as live data.
  • Creating a new client per request.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is writing one point per request slow?

    Fixed per-request costs dominate when requests carry little data.
  2. 2. Which response should NOT be retried?

    A 400 means the data itself is invalid, so retrying the same body fails again.
  3. 3. What does jitter add to exponential backoff?

    Jitter spreads retries and avoids a thundering herd.
  4. 4. Your program ends right after the last write call and some points are missing. Likely cause?

    Batched points sit in a buffer until a flush, which closing the writer triggers.

Summary

  • Batch writes: thousands of points per request, plus a flush interval.
  • Retry transient errors with exponential backoff and jitter; never retry 4xx data errors.
  • Compress bodies, use the coarsest honest precision, and write mostly in time order.
  • Reuse clients and connections; flush on exit.
  • Plan for backpressure with bounded buffers and a drop counter.

Next: Scaling and Architecture