Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 37 ยท Joins, Groups and Windows

group_by and Aggregations

Summarize data per group with Polars group_by and agg. Learn multiple aggregations, filters inside agg, group order, dynamic time windows and group-level tricks.

"Total revenue per region", "average rating per product", "how many orders each customer placed": all of these are group-by questions. You split the table into groups by one or more key columns, compute something for each group, and collect the results in a new table with one row per group.

In Polars a group-by has two parts: df.group_by(keys) creates the groups, and .agg(...) lists the expressions that run inside each group. Because agg takes expressions, everything you learned in the expressions chapter (filters, conditions, string and date functions) works inside it.

Your first group_by

Python Revenue per region

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Run it twice and the row order may change. Polars does not promise any group order, because it processes groups in parallel and in whatever order is fastest. To get a stable result either sort afterwards or pass maintain_order=True:

Python Stable group order

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Sorting the result is almost always better than maintain_order=True. The flag costs speed and only preserves the accident of input order, while a sort expresses what you actually want.

Many aggregations at once

Inside agg you can write as many expressions as you like. Name each with .alias() or with a keyword argument. The useful aggregations are:

Expression Meaning
.sum(), .mean(), .median() Typical statistics
.min(), .max() Extremes
.std(), .var() Spread
.first(), .last() First / last value in each group
.n_unique() Number of distinct values
.count() / pl.len() Non-null count / row count
.quantile(0.9) Any percentile
pl.col("x") alone Collect the group's values into a list
Python Keyword names and many statistics

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The rep_names column holds a list for each group. List columns are normal in Polars and you can work with them using the .list namespace, or flatten them back with explode.

Filtering inside an aggregation

Because agg takes expressions, you can filter inside a group, so different aggregations see different rows. This replaces a lot of awkward "filter, group, join back" code.

Python Conditional aggregation with filter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(pl.col("revenue") > 30) is a boolean column. Summing booleans counts the True values, and their mean is the fraction of True values. That pattern, "count how many rows satisfy a condition per group", is one of the most useful in all of data analysis.

Grouping by several keys and by expressions

Pass a list for several keys. You can also group by an expression, such as the year of a date or the first letter of a name, without creating a helper column first.

Python Multiple keys and computed keys

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Group-level slicing: head, tail, top-N

Inside agg, head(n), tail(n) and sort_by work on each group, which gives you "top 2 sales per region" in one expression. The result is a list column that you can explode.

Python Top two sales per region

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

group_by(...).head(n) and .tail(n) are shortcuts that return rows rather than a list column. For anything more elaborate use a window function with over.

Time-based groups: group_by_dynamic

Grouping timestamps into windows ("per day", "per 3 hours", "per month") is so common that Polars has a dedicated method. The data must be sorted by the time column.

Python Daily and two-day totals

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Days without any rows (here January 3) are missing from the one-day result. You can fill such gaps by joining onto a complete calendar made with pl.date_range. Windows can be "1mo", "1w", "6h" and so on, and you can add period, offset, closed and a group_by= key for per-group windows.

Common mistakes

  • Expecting a fixed row order. Sort the result.
  • Using a Python function per group. agg with built-in expressions is vastly faster than map_elements.
  • Forgetting that nulls are skipped. sum, mean, min and max ignore nulls; pl.len() counts them, pl.col("x").count() does not.
  • Grouping a float column. Floating-point keys group exactly, so 0.1 + 0.2 and 0.3 end up in different groups.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does pl.len() return inside agg?

    pl.len() counts all rows in the group, including rows where some columns are null.
  2. 2. Which expression gives the fraction of orders above 30 in each group?

    A boolean column behaves as 0 and 1 when aggregated, so its mean is the share of True values.
  3. 3. Why might group_by results appear in a different order each run?

    Polars makes no guarantee about the order of groups. Sort the output or set maintain_order=True when order matters.
  4. 4. What must be true of the time column before group_by_dynamic?

    Dynamic windows are built by scanning the column in order, so the data must be sorted by it (Polars has no index).

Summary

  • df.group_by(keys).agg(expressions) returns one row per group.
  • Name outputs with .alias() or keyword arguments.
  • Filter inside agg to aggregate different subsets in a single pass.
  • Boolean columns sum to counts and average to fractions.
  • Group order is not guaranteed: sort the output.
  • group_by_dynamic buckets sorted time columns into windows.

Next: Window Functions with over()