Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 37 ยท Getting Started

Core Contexts: select, filter, with_columns

Learn Polars contexts: select, with_columns, filter, group_by and sort, how each one evaluates expressions, and how to chain them into readable pipelines.

In Polars you rarely loop over rows. Instead you hand expressions to a context, a DataFrame method that knows how to apply them. A context decides what the expressions mean: pick these columns, add those columns, keep rows where this is true, compute this per group. Learn the four or five main contexts and you can already express most data work.

Throughout this lesson we use one small table of orders.

The sample data

Python The orders table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

select: choose or compute columns

select returns only the columns you list. They can be existing columns (by name) or new ones computed by expressions.

Python select

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two things to notice. First, pl.col("price") * pl.col("qty") is arithmetic on whole columns; it runs for every row at once. Second, alias names the result; without it the new column would inherit the name of the left-most column (price).

with_columns: keep everything, add or replace

with_columns works like select but keeps all existing columns, adding new ones or replacing columns of the same name.

Python with_columns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All expressions inside one with_columns are evaluated against the original frame, in parallel. So a column created in the call cannot be used by another expression in the same call. To build on a new column, chain a second with_columns.

A frequent mistake is to write df.with_columns(pl.col("a") * 2) and then print df. The original is unchanged, because the method returns a new DataFrame. Always capture the result: df = df.with_columns(...).

filter: keep rows

filter keeps rows where a boolean expression is true. Combine conditions with & (and), | (or) and ~ (not), and wrap each comparison in parentheses because of Python's operator precedence.

Python filter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Never use Python's and, or, not between expressions. They try to turn an expression into a single True/False and fail. Use &, | and ~.

group_by and agg: summarize per group

group_by splits rows by one or more key columns; agg reduces each group to one row using aggregating expressions.

Python group_by and agg

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The order of groups in the result is not guaranteed unless you pass maintain_order=True to group_by, which is why this example ends with an explicit sort. Joins, groups and windows get their own chapter later; this is only a first taste.

sort and other helpers

sort orders rows by one or more columns. A few simple methods are not expressions at all but still worth knowing: unique, drop, rename, head, limit, reverse. Together with the contexts above they cover the first week of any data job.

Chaining into a pipeline

Because every method returns a DataFrame, you can chain them. Wrapping the chain in parentheses lets you put each step on its own line.

Python
result = (
    orders
    .filter(pl.col("qty") > 1)
    .with_columns((pl.col("price") * pl.col("qty")).alias("total"))
    .select("customer", "total")
    .sort("total", descending=True)
)

Read it top to bottom as a recipe: take the orders, keep multi-unit rows, add a total, keep two columns, sort. This style is the heart of Polars code, and lazy mode (see Lazy Frames: the Basics) optimizes exactly such chains.

Which context should I use?

I want to... Use
Get only some columns, or computed ones select
Add or change columns, keep the rest with_columns
Keep some rows filter
Summarize per group group_by(...).agg(...)
Order rows sort

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the difference between select and with_columns?

    select returns exactly the columns you ask for. with_columns returns the original columns plus any new or replaced ones.
  2. 2. How do you write 'price above 3 and qty at least 2' inside filter?

    Combine boolean expressions with & and wrap each comparison in parentheses. Python's 'and' does not work on expressions.
  3. 3. After orders.with_columns(pl.col('qty') * 2), what is true about orders?

    Polars methods return new DataFrames. Without assignment the result is thrown away.
  4. 4. Can one with_columns call use a column created earlier in the same call?

    All expressions in one call are evaluated against the input frame, in parallel. Chain another call to build on new columns.

Summary

  • Contexts (select, with_columns, filter, group_by().agg(), sort) apply expressions to a DataFrame.
  • select returns only what you ask for; with_columns keeps the rest.
  • Combine conditions with &, |, ~, and parenthesize each comparison.
  • Results are always new frames; chain methods in parentheses for readable pipelines.

Next: Series Basics