Skip to content
Tutorials Center
My learning

Topics

Lesson 16 of 37 ยท Expressions

List and Struct Columns

Work with nested data in Polars: List columns, the list namespace, explode, Struct columns, unnest, field access, building structs and per-row list aggregation.

Real data is not always flat. A customer has several phone numbers, an order has a list of items, an API returns objects inside objects. Polars handles this with two nested types: List (a variable-length list of one type in each cell) and Struct (a group of named fields in each cell). Both stay fast because they use the same columnar storage.

List columns

Python Creating and inspecting lists

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The list namespace works per row on the list inside each cell. An empty list has length 0, sum 0, and a null mean/max.

More list methods

Python The list namespace

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

list.get(i) is positional (negative counts from the end); use list.get(5, null_on_oob=True) to get null instead of an error for short lists. list.eval(expr) runs a full expression on each list's elements, using pl.element() for the current element:

Python list.eval

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

explode: one row per list item

explode unrolls a list column so each item gets its own row, and repeats the other columns. It is the standard way to move from nested to flat.

Python explode and implode

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

An empty list becomes a single row with null after explode, and a null list also becomes null. Filter nulls out if you do not want them. The reverse, building lists, is what agg(pl.col("x")) does: an aggregation that does not reduce returns a list of the group's values.

Aggregating into lists

Python Group values into lists

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Struct columns

A struct packs several named fields into one column. You get structs from pl.struct, from nested dictionaries, from value_counts, and from reading nested JSON.

Python Structs: create, access and unnest

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

struct.field(name) extracts one field as a normal column, unnest expands all fields into columns, and pl.struct(...) packs columns together. Structs are useful for grouping related values, for returning multiple results from one expression, and for comparing multiple columns as a unit (for example is_duplicated on a struct finds duplicate pairs).

Python Struct tricks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Nested data from dictionaries and JSON

When a DataFrame is built from nested Python data, Polars infers List and Struct types automatically, as you saw above. Reading a JSON file with objects and arrays produces the same nested columns, and unnest plus explode flatten them in two steps. A common recipe for an array of objects:

Python
flat = (
    df
    .explode("items")          # one row per element
    .unnest("items")           # object fields become columns
)

Prefer built-in list.* and struct.* methods over map_elements. They operate on the nested data directly, in compiled code, and keep the parallel speed.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does df.explode('items') do?

    explode unrolls a list column into rows. The other columns are repeated for each item.
  2. 2. Which expression gets the number of items in each list of column 'scores'?

    The list namespace works per row. list.len() gives the length of each list.
  3. 3. How do you turn the fields of a struct column 'contact' into ordinary columns?

    unnest expands each struct field into its own column. explode is for lists.
  4. 4. What happens to an empty list when you explode its column?

    Empty lists (and null lists) produce a single row with a null item. Drop nulls afterwards if you do not want them.

Summary

  • List columns hold a variable-length list per row; the list namespace works on each row's list, and list.eval runs full expressions on elements.
  • explode flattens lists into rows; aggregating without reducing builds lists.
  • Struct columns pack named fields; struct.field, unnest and pl.struct read and build them.
  • Use built-in nested-type methods instead of map_elements.

Next: Lazy Frames: the Basics