Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 37 ยท Projects and Practice

Practice: Harder Polars Challenges

Ten harder Polars problems with runnable solutions and explanations: keep latest rows, streaks, tag counts, running balances, top-N per group, forward fill, pair finding and more.

These ten problems come up again and again in real data work and in technical interviews. Each is a small, self-contained puzzle with a data set, the idea behind the solution, and a runnable answer. Read the task, pause and try it in the editor, then read the explanation. The patterns here (flag then cumulative sum, rank then filter, sort then shift) are the building blocks you will reuse for the rest of your Polars life.

Challenge 1: keep the latest row per key

Task: A table holds several versions of each customer record. Keep only the most recent version of each customer.

Python Challenge 1: latest version per customer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

unique(keep="last") keeps the final row of each key in the current order, so sort first. The window version is more explicit, but breaks if two rows share the same latest date.

Challenge 2: longest streak (gaps and islands)

Task: Given daily results (True = the build passed), find the longest run of consecutive passes.

Python Challenge 2: longest streak

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The recipe is flag the changes, cumulative-sum the flags: every change increments a counter, so each run gets its own id. This "gaps and islands" trick solves streaks, sessions and consecutive-date problems. (rle_id() is a built-in shortcut for exactly this run-length id.)

Challenge 3: count tags

Task: Each article has a comma-separated tags string. Count how many articles use each tag, most popular first.

Python Challenge 3: tag counts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Split makes a list, explode makes one row per tag, strip_chars fixes the stray space in " ops". Counting n_unique of the article ids protects against a tag listed twice on one article.

Challenge 4: running balance that resets

Task: Compute a running total of amount per account, but reset to zero whenever a row has reset=True.

Python Challenge 4: resetting running total

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The same flag-and-cumulative-sum idea: each reset starts a new block, and the running sum is computed inside each (account, block) pair. Note that the row with reset=True belongs to the new block, so it starts the new total with its own amount.

Challenge 5: top-N per group, with ties

Task: For each department return the two highest-paid employees; if several people tie for second place, return all of them.

Python Challenge 5: top 2 per group including ties

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

dense ranking gives tied people the same rank and doesn't skip numbers, so "rank at most 2" means "the top two distinct pay levels". With "ordinal" you would get exactly two rows per group and break ties arbitrarily. Which one is right is a business question, so always clarify how ties should behave.

Challenge 6: forward fill within groups

Task: Sensors report a value only when it changes. Produce a table where each row carries the most recent known value of its own sensor.

Python Challenge 6: forward fill per sensor

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Without .over("sensor"), sensor b's first row would be filled from sensor a's last reading, a nasty silent bug. Inside groups, the fill never crosses the boundary.

Challenge 7: compare each month to the same month last year

Task: Given monthly sales over two years, add the percentage change versus the same month of the previous year.

Python Challenge 7: year-over-year

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Grouping by month and sorting by year means "the previous row inside this month's group" is "the same month last year". The 2024 rows have no previous year, so they stay null.

Challenge 8: pairs that share an attribute

Task: Find all pairs of employees who work in the same city, listing each pair once.

Python Challenge 8: pairs in the same city

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A self-join pairs every row with every row of the same city, including itself and each pair twice (A,B and B,A). Requiring name < name_b keeps one copy of each pair and removes the self-pairs. Watch the size: a city with 1,000 people produces about half a million pairs.

Challenge 9: first occurrence flag and running distinct count

Task: For a stream of page visits, mark the first time each user appears, and compute how many distinct users have been seen so far.

Python Challenge 9: new users over time

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

is_first_distinct() is true for the first occurrence of each value. Cumulative-summing that flag gives a running count of distinct users in one pass, with no self-join or loop.

Challenge 10: conditional totals in one pass

Task: From an orders table, report per customer: total orders, number of returned orders, the revenue of non-returned orders, and the date of the last order, all in one query.

Python Challenge 10: a one-pass customer summary

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Customer c has a kept revenue of 0.0: the sum of an empty selection is zero, not null. Know which you want: use .filter(...).sum() for zero, or pl.when(...).then(...).sum() if you prefer to detect "no matching rows" separately.

Patterns worth memorizing

Pattern Recipe
Latest per key sort then unique(keep="last") or filter(col == col.max().over(key))
Streaks / islands / sessions flag changes (!= shift), cum_sum the flags
Top-N per group rank(...).over(group) then filter
Previous value in a group shift(1).over(group) after a sort
Distinct so far is_first_distinct() then cum_sum
Pairs self-join and a < b
Conditional aggregate col.filter(cond).agg() inside agg
Fill inside groups forward_fill().over(group)

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How do you give every run of equal consecutive values its own id?

    Each change increments a counter, so a run keeps one id. Grouping by value would merge separate runs of the same value.
  2. 2. Why do you need .over('sensor') on forward_fill when filling readings from several sensors?

    over keeps the fill inside each group so values never leak across groups.
  3. 3. In a self-join to find pairs, what does the filter name < name_b achieve?

    A row paired with itself fails the strict inequality, and of (A,B) and (B,A) only the one with the smaller first name survives.
  4. 4. What does pl.col('amount').filter(~pl.col('returned')).sum() return for a group where every row is returned?

    The sum of an empty selection is zero. Use a when/then expression if you need to distinguish 'no rows' from 'zero'.

Summary

  • Most "hard" DataFrame problems are a few primitives in the right order: sort, shift, flag, cumulative sum, rank, filter.
  • Use over to keep computations inside groups: ranks, fills, previous values.
  • Clarify tie behaviour (dense, min or ordinal) before ranking.
  • Self-joins are powerful but quadratic: filter early and watch group sizes.
  • Practice with your own data: the same patterns appear everywhere.

Next: Back to the Polars track