Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 39 ยท Performance and Scaling

Scaling and Architecture Choices

Plan InfluxDB capacity from write rate and point size, understand scale-up and scale-out options, separate ingest and query, and know when to add a queue or edge tier.

Scaling a time-series system is mostly arithmetic before it is engineering. If you know how many points per second arrive, how big a point is after compression, and how long you keep it, you can estimate disk, memory and CPU long before the first server is bought. This lesson gives you that arithmetic, then lays out the architecture options from a single node to a distributed deployment.

The exact deployment features differ per edition and change over time, so this lesson stays at the level of principles, and you should check the product documentation for the version you run. The calculators below are runnable stand-ins; no InfluxDB server is involved.

Step 1: estimate the load

Gather three numbers:

  1. Series count (see cardinality).
  2. Write interval per series.
  3. Values per point (fields).

Then:

  • Points per second = series x fields / interval
  • Raw bytes per day = points per second x 86,400 x bytes per point

A line-protocol point of about 100 to 150 bytes becomes much smaller on disk: time-series compression (delta timestamps, floating-point and run-length encodings, Parquet and zstd in InfluxDB 3) often reaches a ratio of 5x to 20x or more. Measure with your own data by loading a sample, because ratio depends strongly on how regular the values are.

JS Stand-in: capacity planner

Press Run (or Ctrl + Enter) to see the console output here.

No output.

All inputs are assumptions you must replace with measurements. The goal is the order of magnitude: gigabytes, hundreds of gigabytes, or terabytes. Then add headroom of at least 30 to 50 percent for compaction, snapshots and growth.

Step 2: scale up first

Before building a cluster, a single well-sized node goes a long way:

  • Fast local SSD or NVMe storage; time-series databases are write- and read-heavy on disk.
  • Enough RAM to cache recent data and, in 1.x/2.x, the series index.
  • Several CPU cores for compaction and parallel queries.
  • Separate disks (or volumes) for data and write-ahead log, where the version supports it.
  • Reduce load through batching, downsampling and lower cardinality, which is cheaper than hardware.

Step 3: scale out

Options depend on the product line:

  • InfluxDB 1.x / 2.x open source runs as a single node. You scale by sharding data across several independent instances yourself, for example by customer, by region, or by hashing a key, and querying across them in your application or dashboard tool.
  • InfluxDB Enterprise (1.x) and InfluxDB Cloud offer clustering or managed scaling.
  • InfluxDB 3 separates ingest, query, compaction and storage: object storage (such as S3) holds Parquet files, and different nodes handle writes and reads, so you can scale readers independently of writers. Which of these roles you can run depends on the edition, so read the documentation of the one you use.
flowchart LR
  A["Agents<br/>Telegraf, apps"] --> Q["Queue or load balancer"]
  Q --> I["Ingest nodes"]
  I --> O[("Object storage<br/>Parquet files")]
  C["Compactor"] --> O
  R["Query nodes"] --> O
  D["Dashboards<br/>and alerts"] --> R

A separated architecture lets writers, compactors and readers scale on their own.

The hash-sharding idea

If you shard by hand, you pick a shard key and route every series to the same node. A hash of the key chosen modulo the number of nodes spreads load evenly, and the same series always goes to the same place, so a series is never split.

JS Stand-in: route series to nodes by hash

Press Run (or Ctrl + Enter) to see the console output here.

No output.

That second number is why real systems use consistent hashing or a lookup table: adding a node should move only a small share of the data. It is also why you should plan the number of shards up front, with room to grow.

Separate concerns

  • Write path vs query path. A heavy dashboard query should not slow ingest. Use read replicas or separate query nodes where available, and cap query time and memory.
  • Hot vs cold data. Keep recent raw data on fast storage and long-term rollups on cheaper storage (downsampling).
  • Buffering. Put a queue (Kafka, a message bus) or Telegraf's buffer between sources and the database so that a database restart does not lose data.
  • Edge aggregation. Aggregate at the source to cut volume across slow links.

High availability and backup

  • Back up regularly and test restores. Use the built-in backup and restore tools of your version.
  • Replicate writes to two destinations (Telegraf supports several outputs) if you need a warm standby.
  • Monitor the monitor. Scrape InfluxDB's own metrics (write errors, query durations, queue depth, disk free) and alert on them.
  • Put the database behind TLS, with tokens scoped by bucket or database.

Common mistakes

  • Buying a cluster before fixing cardinality or batch size.
  • Sizing disks from raw bytes while ignoring compaction headroom.
  • No buffer in front of the database, so an upgrade loses minutes of data.
  • Hand-sharding with hash % n and then adding nodes without a migration plan.
  • Never testing a restore.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Points per second for 10,000 series, one field each, written every 10 seconds?

    10,000 series x 1 field / 10 s = 1,000 points per second.
  2. 2. What should you try before building a cluster?

    These changes are cheap and often remove the need to scale out.
  3. 3. Why is hash modulo n a risky sharding scheme when n changes?

    Changing the modulus remaps most keys. Consistent hashing limits the movement.
  4. 4. What is the benefit of a queue between agents and the database?

    A buffer absorbs data while the database is unavailable or slow.

Summary

  • Estimate points per second, bytes per day and total retention from series, interval and fields.
  • Scale up and fix cardinality and batching first.
  • Scale out by sharding (open source 1.x/2.x), clustering or separated roles (3.x); check your edition.
  • Separate ingest from query, use buffers, and plan for backup and monitoring.
  • Hash sharding needs a plan for growth.

Next: Telegraf Pipelines