Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 37 ยท Queries and Transformations

Grafana Transformations: Join, Group, Calculate and Reshape Data

Reshape query results in Grafana with transformations: merge, join, group by, reduce, calculate field, filter, organize fields and convert types, with a runnable pipeline stand-in.

Queries return data in the shape the data source likes. Panels need it in the shape the visualization likes. Transformations close that gap inside Grafana, after the data arrives and before it is drawn. They run in the browser on the data frames, as an ordered pipeline: the output of one step is the input of the next.

Grafana is not available on this site, so the menu steps are described in words. The runnable blocks are a Python stand-in that implements a few transformations on plain lists, to show what each one does to the rows. It is a model for learning, not Grafana's code.

Where to find them

Edit a panel and open the Transformations tab, then click Add transformation and search by name. Each added step becomes a card; you can drag the cards to reorder them, disable one with the eye icon (useful for debugging) and delete it. Many panels start with a hint, such as "Table: use Organize fields to hide columns". The Query inspector > Data tab can show data "with transformations applied" so you can inspect the result.

Because transformations run on the client, they are fine for thousands of rows but not for millions. Heavy joins and aggregations belong in the query itself (SQL GROUP BY, PromQL sum by). Use transformations to tidy, combine and compute on results of a sensible size.

The most useful transformations

Transformation What it does
Organize fields by name Hide, rename and reorder columns in one card.
Rename by regex Rename many fields with a regex and replacement.
Filter data by values Keep or drop rows matching conditions on a field.
Filter fields by name Keep or drop columns by name or regex.
Sort by Sort rows by a field, ascending or descending.
Limit Keep only the first N rows.
Reduce Collapse each series or field to its calculations (Last, Max, Mean, Total, Count...).
Group by Group rows by a field and compute aggregates per group.
Merge Combine several queries' frames into one table by stacking rows.
Join by field Combine frames side by side on a shared field (inner or outer join).
Add field from calculation (Calculate field) Add a column computed from other columns, such as A / B.
Convert field type Change a field to time, number, string, boolean.
Labels to fields Turn series labels into columns.
Prepare time series Convert between wide and long time series layouts.
Partition by values Split one frame into several by the values of a field.
Series to rows / Concatenate fields Further reshaping helpers.

Names change slightly across Grafana versions (for example Calculate field was renamed Add field from calculation), and new ones appear regularly, so use the search box.

A worked example

Imagine two queries on the same panel:

  • Query A (PostgreSQL, table): host, requests.
  • Query B (PostgreSQL, table): host, errors.

You want a table with host, requests, errors and an error percentage, sorted worst first and hiding anything with fewer than 100 requests. The pipeline is:

  1. Join by field: field host, mode OUTER (so hosts missing from one query still show).
  2. Add field from calculation: mode Binary operation, errors / requests, alias error_rate.
  3. Filter data by values: field requests, condition Greater than 99.
  4. Sort by: error_rate, reverse (descending).
  5. Organize fields: rename error_rate to Error rate; then in Standard options set the unit to Percent (0.0-1.0).
Python Stand-in: a transformation pipeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that host e (present only in the errors query) is removed by the filter, and host b (only 80 requests) is removed as well. Each step can be inspected by printing its variable, exactly like you can disable later cards in Grafana to see the intermediate table.

Merge versus Join

These two are easy to mix up.

  • Merge stacks frames that have the same columns into one: A returns (time, cpu) for server 1 and B returns (time, cpu) for server 2; merge gives one table with all rows. Use it to make one table out of several similar queries.
  • Join by field puts frames next to each other matching a key: A has (host, requests) and B has (host, errors); join gives (host, requests, errors). For time series, join by Time to align several series into one table.

If a table shows duplicated rows or lots of empty cells, you probably chose the wrong one.

Reduce and Group by

Reduce has two modes. Series to rows turns each series into one row with columns for each calculation (Min, Max, Mean, Last, Total...). This is the standard way to make a "min / max / avg" table from a time series graph query. Reduce fields keeps the frame and replaces each field by its calculation.

Group by works on rows: pick one field as Group by, and choose Calculate (Total, Mean, Count, Last...) on other fields. The result has one row per distinct group value. It is the client-side version of SQL GROUP BY.

Python Stand-in: Group by with Total and Mean

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Ordering matters

Transformations apply from top to bottom. Typical order: filter early (fewer rows to process), join or merge, calculate, group or reduce, sort, then organize (rename and hide) last. If a later card cannot find a field, an earlier card probably renamed or removed it.

You can also save a transformation as a library entry in newer versions, or copy panel JSON between dashboards; the transformation list lives under transformations in the panel's JSON model, which you will meet again in dashboards as code.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Where do transformations run?

    They process the frames after the query returns and before the visualization draws them.
  2. 2. Which transformation puts two frames side by side matching a shared host field?

    Join by field aligns frames on a key; Merge stacks frames with the same columns.
  3. 3. You have a time series and need a table of min, max and mean per series. What do you use?

    Reduce in Series to rows mode creates one row per series with columns for each calculation.
  4. 4. Why should heavy aggregation over millions of rows not rely on transformations?

    Client-side processing is fine for moderate sizes; push big aggregations into SQL or PromQL.

Summary

  • Transformations are an ordered client-side pipeline on the panel's data frames.
  • Learn Organize fields, Filter data by values, Sort by, Reduce, Group by, Merge, Join by field and Add field from calculation first.
  • Merge stacks rows; Join puts columns side by side on a key.
  • Filter early, rename late, and disable cards to debug.
  • Large aggregations belong in the query, not in the browser.

Next: Expressions and Mixed Queries