Skip to content
Tutorials Center
My learning

Topics

Lesson 16 of 39 ยท Querying: Flux and SQL

Flux Basics: Pipelines, Tables and Functions

Learn the Flux query language of InfluxDB 2.x: from, range, filter, the pipe-forward operator, the table stream model, variables, functions and the _time and _value columns.

Flux is the functional query and scripting language of InfluxDB 2.x. Where SQL describes what you want, a Flux script describes how data flows: you start with a source, then pass it through a chain of functions with the pipe-forward operator |>. Flux can query, transform, join, call external systems and write results to another bucket, which is why 2.x tasks and dashboards use it everywhere.

Flux runs only inside InfluxDB 2.x (and compatible Cloud products). InfluxDB 3 uses SQL and InfluxQL instead, and Flux is in maintenance mode, so learn it mainly for existing 2.x systems. The Flux code in this lesson is read-only. The runnable blocks are JavaScript stand-ins that mimic the pipeline idea with the same function names, so you can feel the flow.

The shape of a query

TEXT
from(bucket: "weather")
  |> range(start: -6h)
  |> filter(fn: (r) => r._measurement == "weather" and r._field == "temp" and r.city == "Lisbon")
  |> aggregateWindow(every: 30m, fn: mean, createEmpty: false)
  |> yield(name: "temp_mean")

Read it top to bottom as a pipeline:

  1. from(bucket: ...) produces a stream of tables from storage.
  2. range(start: ...) keeps only rows in the time window. Every query needs a range; without it the query is rejected, which protects you from scanning the whole bucket.
  3. filter(fn: ...) keeps rows for which the function returns true.
  4. aggregateWindow groups rows into 30-minute windows and takes the mean of each.
  5. yield(name: ...) names the result (needed when a script returns several).
flowchart LR
  A["from"] --> B["range"] --> C["filter"] --> D["aggregateWindow"] --> E["yield"]

Each function takes the stream from the previous one: the pipe operator is plain function composition.

The data model of a Flux result

Flux does not return a single flat table. It returns a stream of tables. Each table contains rows that share the same group key, by default the set of _measurement, _field and all the tag values (plus _start and _stop). Every row has standard columns:

Column Meaning
_start, _stop The query's time range (same in every row)
_time Timestamp of the point
_measurement The measurement name
_field The field key
_value The field value
tag columns One per tag key, e.g. city, sensor

Notice that fields are stored one row per field: a point with temp and humidity becomes two rows, one with _field = "temp" and one with _field = "humidity". That is why most queries filter on _field, and why you later use pivot() to get one column per field (Windows, Gaps and Pivot).

Functions and lambdas

fn: (r) => ... is a lambda: an anonymous function that receives a record r (one row). Access columns with dot notation (r._value, r.city) or bracket notation (r["my-tag"]) when the name contains special characters.

Comparison operators are ==, !=, <, >, <=, >=; logical operators are and, or, not; regex matching uses =~ and !~.

TEXT
|> filter(fn: (r) => r._measurement == "cpu" and r.host =~ /^web-/ and r._value > 80.0)

Variables, functions and imports

TEXT
import "math"
import "strings"

threshold = 80.0
bucketName = "metrics"

hot = (tables=<-, limit) => tables
  |> filter(fn: (r) => r._value > limit)

from(bucket: bucketName)
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage")
  |> hot(limit: threshold)

tables=<- declares the parameter that receives the piped stream, so your own functions can sit in a pipeline just like built-ins. Dashboard variables arrive as v.bucket, v.timeRangeStart and v.windowPeriod, which the UI fills in.

Flux is typed: 80 is an integer and 80.0 a float, and comparing a float field with an integer literal fails with a type error. Write floats with a decimal point.

A pipeline in plain JavaScript

This stand-in re-creates range, filter and aggregateWindow over rows shaped like Flux output. It is not Flux, but it shows precisely what each stage does to the data.

JS range, filter and aggregateWindow imitated (stand-in)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Flux's aggregateWindow labels each window by its stop time by default, which is why the output times are the end of each window. You can change that with timeSrc: "_start".

Why a query returns many tables

Because each distinct tag combination is its own group, a query over many hosts returns many tables. mean() then produces one value per table, i.e. per host. To get one overall number, merge the tables first with group():

TEXT
|> group()                  // one single table
|> mean()

|> group(columns: ["region"]) // one table per region
|> mean()

The difference between "mean per host" and "mean over all hosts" is the most common source of confusion for beginners. Always ask: which group key do I have at this point?

JS group changes what an aggregate means (stand-in)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Useful building blocks

Function Purpose
range, filter, limit, sort Select and order rows
first, last, min, max, mean, sum, count, median Aggregates (one row per table)
aggregateWindow Aggregate in time windows
group, keep, drop, rename, map Shape columns and groups
pivot Turn field rows into columns
derivative, difference Rates of change for counters
join, union Combine streams
to Write results to another bucket (downsampling)

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the |> operator do in Flux?

    Pipe-forward passes the stream of tables on the left as input to the function on the right.
  2. 2. Which function is mandatory in a Flux query after from()?

    Flux requires a time range so that queries never scan a whole bucket by accident.
  3. 3. A point has fields temp and humidity. How does Flux return it?

    Flux uses a narrow layout: one row per field. Use pivot() to widen it.
  4. 4. What does group() with no arguments do?

    An empty group key puts every row into one table, so later aggregates cover everything.

Summary

  • A Flux script is a pipeline: from |> range |> filter |> ... |> yield.
  • Results are streams of tables grouped by a group key; fields come as rows with _field and _value.
  • Lambdas (r) => ... select and transform rows; Flux is strongly typed.
  • group() decides what an aggregate covers; aggregateWindow summarises per time window.
  • Flux belongs to 2.x; 3.x uses SQL and InfluxQL.

Next: Flux: Filtering, Ranges and Aggregates