Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 39 ยท Integrations

Prometheus and Metrics Bridges

Move Prometheus-style metrics into InfluxDB with Telegraf, understand counters, gauges and histograms, and compute per-second rates safely, including counter resets.

Prometheus is the dominant metrics system in container and Kubernetes environments, and almost every modern service exposes metrics in its text format at a /metrics endpoint. InfluxDB and Prometheus overlap in purpose, and many teams run both, or want Prometheus-style metrics stored in InfluxDB for longer retention, SQL queries or joining with IoT data.

This lesson explains the Prometheus data model in InfluxDB terms, shows how Telegraf bridges the two (read-only configuration), and then lets you run the two key conversions yourself: parsing the exposition format, and computing a correct per-second rate from a counter that resets.

The data model, side by side

Prometheus InfluxDB
Metric name (http_requests_total) Measurement or field name
Labels (method="GET") Tags
Sample value Field value
Timestamp Timestamp

A Prometheus time series is a metric name plus labels, much like an InfluxDB series. The same warning about cardinality applies to labels.

Metric types matter because they decide how you query them:

  • Counter: only goes up (resets to zero on restart). Examples: requests served, bytes sent. You almost never chart the raw value; you chart its rate of change.
  • Gauge: goes up and down. Examples: memory in use, queue length, temperature. Chart it directly.
  • Histogram: counters for observations falling into buckets (_bucket{le="0.5"}), plus _sum and _count, used to estimate percentiles.
  • Summary: pre-computed quantiles, _sum and _count.

How to bridge them

Three approaches:

  1. Telegraf scrapes Prometheus endpoints with the inputs.prometheus plugin and writes to InfluxDB. This is the simplest way to store a service's /metrics in InfluxDB. Read-only example:
TOML
[[inputs.prometheus]]
  urls = ["http://app-1:9100/metrics", "http://app-2:9100/metrics"]
  metric_version = 2
  response_timeout = "5s"

[[outputs.influxdb_v2]]
  urls = ["https://influx.example.com:8086"]
  token = "${INFLUX_TOKEN}"
  organization = "my-org"
  bucket = "prometheus"

With metric_version = 2 each metric family becomes a measurement named after the input (prometheus), the metric name goes into a field, and labels become tags. Check the plugin documentation for the layout of your Telegraf version.

  1. Prometheus remote write. Prometheus can forward samples to a compatible receiver. Telegraf has a receiver input that accepts remote-write data and a parser for it, so Prometheus can feed InfluxDB through Telegraf. Configure it carefully, because remote write sends a high volume.

  2. Query across both in Grafana by adding each as a data source, without copying data. This is the lightest option when you only need joined dashboards.

flowchart LR
  A["Service /metrics"] --> T["Telegraf<br/>inputs.prometheus"]
  A --> P["Prometheus"]
  P -- "remote write" --> T2["Telegraf receiver"]
  T --> I[("InfluxDB")]
  T2 --> I

Two ways to land Prometheus-style metrics in InfluxDB: scrape directly, or receive remote write.

Parsing the exposition format

The text format is easy to read:

TEXT
# HELP http_requests_total Total requests served.
# TYPE http_requests_total counter
http_requests_total{method="GET",code="200"} 1027
http_requests_total{method="POST",code="500"} 3

Each non-comment line is name{labels} value. This runnable parser turns lines into InfluxDB line protocol, which is what a bridge does.

JS Stand-in: Prometheus text to line protocol

Press Run (or Ctrl + Enter) to see the console output here.

No output.

This mirrors the layout described above: the measurement is prometheus, labels are tags, and the metric name is the field key.

Rates from counters

A counter's raw value is useless on a dashboard: it only climbs. What you want is the rate, such as requests per second. Take two samples and divide the difference by the elapsed time. But when a process restarts, the counter resets to zero, and a naive difference goes negative. The standard handling: if the value decreased, treat the new value as the increase since the reset.

SQL Stand-in: per-second rate with counter resets

Press Run (or Ctrl + Enter) to see the result here.

The row at ts = 180 shows a naive increase of -1260 but a safe increase of 40. In InfluxDB Flux, the function derivative(unit: 1s, nonNegative: true) does this, and increase() totals the growth with reset handling; in SQL for InfluxDB 3 you write the LAG logic above, as the real products differ in available functions, so check your version.

JavaScript
// read-only Flux
from(bucket: "prometheus")
  |> range(start: -1h)
  |> filter(fn: (r) => r._field == "http_requests_total")
  |> derivative(unit: 1s, nonNegative: true)

Histograms and percentiles

A Prometheus histogram stores cumulative counts per bucket (le = "less than or equal"). To estimate the 95th percentile, find the bucket where the cumulative count crosses 95 percent and interpolate inside it. Percentiles cannot be averaged: store the buckets and compute the quantile at query time (or keep several quantiles when you downsample). Use sensible bucket boundaries, since a coarse histogram gives coarse percentiles.

When to use which tool

Need Choose
Short-term service alerts in Kubernetes Prometheus and Alertmanager
Long-term storage and SQL over metrics InfluxDB fed by Telegraf
IoT, sensors, mixed push and pull InfluxDB with Telegraf
One dashboard over both Grafana with two data sources

They complement each other more than they compete.

Common mistakes

  • Charting raw counter values.
  • Ignoring counter resets when computing rates.
  • Ingesting every label, including ones with unbounded values (user IDs, full URLs).
  • Scraping too frequently: a 1-second scrape of thousands of series is costly for little benefit.
  • Averaging percentile values from different instances.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What should you chart for a Prometheus counter such as http_requests_total?

    A counter only grows, so the useful signal is how fast it grows.
  2. 2. A counter goes from 1300 to 40 between samples. What most likely happened?

    Counters reset to zero on restart. The increase since the reset is the new value.
  3. 3. Prometheus labels become what in InfluxDB?

    Labels identify the series, like tags.
  4. 4. Which Telegraf plugin scrapes a /metrics endpoint?

    inputs.prometheus reads Prometheus-format endpoints.

Summary

  • Prometheus metric names, labels and values map to InfluxDB fields, tags and values.
  • Counters need rates with reset handling; gauges are charted directly; histograms estimate percentiles.
  • Telegraf scrapes endpoints or receives remote write to land data in InfluxDB.
  • Watch label cardinality and scrape intervals.
  • Use Grafana to combine both without copying data.

Next: Project: Server Monitoring