Skip to content
Tutorials Center
My learning

Topics

Lesson 11 of 37 ยท Data Sources

The Prometheus Data Source and PromQL Basics

Connect Grafana to Prometheus and learn PromQL essentials: selectors, label matchers, rate, sum by, histogram_quantile, instant versus range queries and the query editor.

Prometheus is the most common metrics source for Grafana. It stores time series and answers questions written in PromQL. This lesson connects the two and teaches the PromQL you need for 90 percent of dashboards.

Neither Prometheus nor PromQL runs on this site. PromQL examples are shown as read-only code. The runnable blocks are stand-ins that compute the same results (rate, sum by, quantiles) on small arrays.

Connecting

Connections > Data sources > Add > Prometheus. Set:

  • URL: http://prometheus:9090 (or the address Grafana can reach).
  • Scrape interval: match your Prometheus scrape_interval (default 15s). Grafana uses it for $__rate_interval and step minimums.
  • Query timeout and HTTP method (POST helps with very long queries).
  • Authentication if your Prometheus is behind a proxy.

Press Save & test; you want "Successfully queried the Prometheus API".

The metric types

Type Meaning How to query
Counter Only goes up (resets on restart): requests, errors rate() or increase()
Gauge Goes up and down: memory, temperature Use directly, or avg_over_time
Histogram Buckets of observations (_bucket, _sum, _count) histogram_quantile()
Summary Pre-computed quantiles Use directly

Selectors and label matchers

TEXT
http_requests_total
http_requests_total{job="shop"}
http_requests_total{job="shop", status=~"5.."}
http_requests_total{env!="test"}

Matchers: = equals, != not equal, =~ regex match, !~ regex no match. A selector returns an instant vector (one value per series at one time). Adding [5m] makes a range vector (a window of samples), used by functions like rate.

The core functions

TEXT
rate(http_requests_total[5m])                      # per-second rate over 5m
sum(rate(http_requests_total[5m]))                 # all series added together
sum by (status) (rate(http_requests_total[5m]))    # one result per status
avg without (instance) (node_load1)                # drop one label, keep others
increase(http_requests_total[1h])                  # total growth in an hour
histogram_quantile(0.95,
  sum by (le) (rate(http_request_duration_seconds_bucket[5m])))   # p95 latency
topk(5, sum by (path) (rate(http_requests_total[5m])))            # 5 busiest paths

In Grafana, replace the fixed [5m] with [$__rate_interval].

Useful arithmetic: error ratio = sum(rate(errors[5m])) / sum(rate(requests[5m])); format the panel unit as Percent (0.0-1.0).

Counters and rate: stand-in

A counter's raw value is rarely useful; its rate is. Prometheus computes it from the first and last sample in the window (with reset handling and extrapolation). The simplified stand-in handles resets and shows the idea.

JS Stand-in: per-second rate of a counter

Press Run (or Ctrl + Enter) to see the console output here.

No output.

(The real rate() also extrapolates to window edges; the result is close but not identical.)

Aggregation: stand-in for sum by

JS Stand-in: sum by (label)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The Grafana query editor

The Prometheus editor has two modes: Builder (pick metric, labels and operations from menus, and Grafana writes PromQL) and Code (type PromQL with autocomplete and a metrics browser). Important options below the query:

  • Legend: Auto, Verbose or Custom, e.g. {{instance}} - {{status}}.
  • Min step: lower bound for the resolution.
  • Format: Time series, Table or Heatmap.
  • Type: Range (graphs), Instant (one value, good for tables and stat panels) or Both.
  • Exemplars: link metrics to traces if enabled.

Use Explore with the Metrics browser to discover which metrics and labels exist.

Quantiles: stand-in

histogram_quantile interpolates inside cumulative buckets. This simplified version shows why results are estimates tied to your bucket boundaries.

JS Stand-in: quantile from histogram buckets

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Common mistakes

  • Using rate() on a gauge, or graphing a raw counter.
  • Forgetting sum by (le) inside histogram_quantile.
  • A range window shorter than two scrapes: the graph is empty.
  • Wrong URL (localhost) in containers.
  • High-cardinality by clauses that return thousands of series.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which function turns a counter into a per-second rate?

    rate() computes the per-second average increase over a range window.
  2. 2. What does the matcher status=~"5.." do?

    =~ is a regex match.
  3. 3. Which query Type returns one value per series, suited to Stat panels and tables?

    Instant queries evaluate at one moment instead of over the whole range.
  4. 4. Which metric type needs histogram_quantile for percentiles?

    Histograms store bucket counts; histogram_quantile estimates a percentile from them.

Summary

  • Add Prometheus with a reachable URL and the right scrape interval.
  • Counters need rate(); gauges are used as they are; histograms need histogram_quantile.
  • sum by, topk and arithmetic produce the numbers dashboards show.
  • Builder mode helps beginners, Code mode gives full control.
  • Use $__rate_interval and sensible legends.

Next: The Loki Data Source and LogQL Basics