Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 31 ยท Altair and Vega-Lite

Altair Transforms and Aggregation

Learn Altair and Vega-Lite transforms: filter, calculate, aggregate, bin, window and fold. Compare the spec with plain Python code that does the same work.

A chart often needs numbers that are not in your table: a total per region, a running sum, a rounded bucket, a column computed from two others. You could prepare all of this in pandas before charting. Altair gives you another place to do it: transforms, steps that live inside the chart specification and run in the browser each time the chart is drawn.

Why would you want that? A spec that carries its own transforms is self-describing: anyone reading the JSON sees how the numbers were produced. It also keeps the Python code short, and it lets interactive charts re-run the steps when a selection changes (the interaction lesson depends on this). The cost is that the browser does the work, so heavy aggregation on large data belongs in pandas or a database; see the large datasets lesson.

Altair and Vega-Lite cannot run on this site, so the chart code below is read-only. The runnable example is a stand-in: plain Python functions that perform each transform on a list of dictionaries. They show what each step means; the real implementations live in Vega and handle many more options.

Transforms run in order

In a spec, transform is a list. Each step reads the rows left by the step before. The chart's encodings then see the final result. In Altair you add steps with transform_* methods, and the order of your method calls is the order of the steps.

flowchart LR
    R["raw rows"] --> F["filter"] --> C["calculate"] --> G["aggregate<br/>or window"] --> E["encode and draw"]

A typical pipeline: drop rows, add columns, summarize, then draw.

The examples in this lesson use a small sales table with a month number, a region, units sold and a unit price.

The six transforms you will use most

filter

Keeps only the rows for which a condition is true.

Python
import altair as alt

base = alt.Chart(sales)

base.transform_filter("datum.units > 9")                     # expression string
base.transform_filter(alt.datum.units > 9)                   # Python expression
base.transform_filter(alt.FieldOneOfPredicate("region", ["North"]))

The expression language is a small subset of JavaScript where datum is the current row. Strings and alt.datum produce the same JSON.

calculate

Adds a new field computed from each row.

Python
base.transform_calculate(revenue="datum.units * datum.price")

In JSON: {"calculate": "datum.units * datum.price", "as": "revenue"}.

aggregate and joinaggregate

aggregate collapses rows into one row per group. joinaggregate computes the same statistics but keeps every row and attaches the group result to each one, which is how you compare a row with its group's average.

Python
base.transform_aggregate(total="sum(units)", groupby=["region"])
base.transform_joinaggregate(region_avg="mean(units)", groupby=["region"])

The operations include count, sum, mean, median, min, max, stdev, variance and quantile helpers such as q1 and q3.

bin

Turns a continuous number into equal-width buckets. It writes two fields: the start of the bucket and the end.

Python
base.transform_bin("units_bin", field="units", bin=alt.Bin(step=5))

This creates units_bin and units_bin_end. A histogram is exactly "bin, then count rows per bin".

window

Computes a value from neighbouring rows: running totals, moving averages, ranks, previous values. You choose the sort order, an optional grouping and a frame, the span of rows to include relative to the current one.

Python
base.transform_window(
    running="sum(units)",
    sort=[alt.SortField("m")],
    groupby=["region"],
    frame=[None, 0],        # from the start up to the current row
)

frame=[-1, 0] means "the previous row and this one", which gives a two-point moving average with mean(units). Ranking uses operations such as rank() and row_number().

fold

Turns several columns into rows: a wide-to-long reshape. It produces a key column (the old column name) and a value column.

Python
base.transform_fold(["units", "revenue"], as_=["measure", "value"])

This is the Vega-Lite counterpart of melt in pandas, and it is the usual way to draw several measures as separate colored lines. The tidy data lesson explains why this long shape suits plotting.

The same steps in plain Python

Reading a definition is one thing; seeing the data change is better. The stand-in below defines each transform as a short function and applies them to eight rows. Run it, then change a threshold or a frame and run it again.

Python Each transform as a Python function

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the joinaggregate output: every row still exists, and each row has the average of its own region next to it. After aggregate, only one row per region would remain. In the window output, the running total restarts for each region because of the grouping, which is the job of groupby. The bin output shows why a bin needs two fields: a bar in a histogram runs from units_bin to units_bin_end.

One difference from the real tools: Vega's window transform has several more operations (rank, lag, lead, first_value and others) and sorts with a compound sort definition. The stand-in is a model for understanding the frame, not a replacement.

Transforms inside a chart, and shorthand in encodings

You can chain several steps and then draw the result.

Python
chart = (
    alt.Chart(sales)
    .transform_filter("datum.units > 9")
    .transform_calculate(revenue="datum.units * datum.price")
    .transform_aggregate(total_revenue="sum(revenue)", groupby=["region"])
    .mark_bar()
    .encode(x="region:N", y="total_revenue:Q")
)

Simple cases need no transform_* call at all, because an encoding channel can aggregate, bin or truncate dates itself:

Python
alt.Chart(sales).mark_bar().encode(x="region:N", y="sum(units):Q")          # aggregate
alt.Chart(sales).mark_bar().encode(x=alt.X("units:Q", bin=alt.Bin(step=5)), y="count()")  # histogram
alt.Chart(orders).mark_line().encode(x="yearmonth(date):T", y="sum(amount):Q")             # time unit

The shorthand "count()" counts rows. A timeUnit such as yearmonth, month, day or hours groups dates by that unit before the chart is drawn. The shorthand form and the explicit transform produce the same numbers; choose the explicit form when you need the intermediate field for several encodings or layers, or when the step sequence is long enough to deserve its own lines.

When to transform in the chart and when to prepare data first

Situation Better place
Few thousand rows, simple summary Encoding shorthand or a transform
An interactive filter must change the summary Transform in the chart
Hundreds of thousands of rows, or a join with another table pandas, SQL or a database before charting
The same numbers are needed in a table next to the chart Prepare once in Python and chart the result
You want a unit test on the numbers Python, where the test can read them directly

Altair also has a limit by default: it raises an error when the dataset has more than 5,000 rows, because the data is embedded in the spec. This is a protection, not a bug; aggregate first, switch to a URL data source, or raise the limit deliberately.

A second look at the stand-in: composing steps

Transform order matters. Aggregating before filtering is a different question than filtering before aggregating. The next TryIt makes the difference concrete with the same functions in a different order.

Python Order of transforms changes the answer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first pipeline answers "how many units came from large orders?" and the second answers "which regions sold a lot overall?". Both are valid questions; the transform list in the spec records which one the chart answers.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the difference between aggregate and joinaggregate?

    aggregate collapses the data. joinaggregate adds the group statistic as a new field while leaving all rows in place, so a row can be compared with its group.
  2. 2. A bin transform on units with step 5 produces which fields?

    A bin has a start and an end value, so Vega-Lite writes two fields, for example units_bin and units_bin_end.
  3. 3. Which window frame gives a moving average of the previous row and the current row?

    The frame is given relative to the current row. -1 is one row back and 0 is the current row, so [-1, 0] covers two rows.
  4. 4. You have 200,000 raw rows and only need totals per region in the chart. What is the best approach?

    Embedding every row makes the spec large and slow. Summarize before charting unless an interaction needs the raw rows in the browser.

Summary

  • Transforms are ordered steps inside the chart spec; later steps see the output of earlier ones.
  • The core six are filter, calculate, aggregate (and joinaggregate), bin, window and fold.
  • Encoding shorthands such as sum(units):Q, count(), bin and timeUnit cover simple cases without a separate transform.
  • Order changes meaning: filtering before aggregating answers a different question than aggregating before filtering.
  • Heavy data work belongs in Python or a database; use in-chart transforms for small data and for interactions.
  • The Python functions in this lesson are stand-ins that model what each step does.

Next: Layering, Faceting and Concatenation