Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 37 ยท Expressions

String Expressions

Clean and transform text in Polars with the str namespace: case, strip, contains, regex extract, replace, split, slicing, padding, parsing and concatenation.

Text columns need cleaning more often than any other kind: stray spaces, mixed case, codes glued together, numbers stored as text. Polars keeps all text operations in the str namespace of an expression: pl.col("name").str.<method>(). Each method works on the whole column at once, and null stays null.

Python Sample messy data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Case, trimming and length

Python Case, strip and length

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

strip_chars, strip_chars_start and strip_chars_end remove whitespace (or a given set of characters). len_chars counts characters, len_bytes counts UTF-8 bytes: they differ for accented letters and emoji.

Searching: contains, starts_with, ends_with, find

Python Searching text

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

contains interprets its pattern as a regular expression by default. Pass literal=True to match the exact characters, which is faster and avoids surprises with dots and brackets.

Extracting and replacing

Python Regex extract and replace

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

  • extract(pattern, group_index) returns one capture group of the first match (or null).
  • extract_all returns a list column of every match.
  • replace changes the first match, replace_all all of them. Both are regex by default; use literal=True otherwise. In the replacement, $1 refers to a capture group.
  • str.replace_many swaps many fixed strings at once, quickly.

Splitting and joining

Python Split and join

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

split returns a list column. split_exact(by, n) returns a struct of n+1 fields, convenient for fixed formats, and unnest turns the struct fields into columns. See List and Struct Columns for what you can do with them.

Slicing, padding and concatenation

Python Slice, pad and concatenate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

pl.concat_str glues columns (and literals) together; the separator goes between parts. pl.format is its template form. Note the + operator also concatenates strings: pl.col("first") + " " + pl.col("last"), but unlike concat_str it returns null if any part is null (by default concat_str does the same; pass ignore_nulls=True to skip them).

Parsing text into other types

Numbers and dates often arrive as text. Cast, or use the dedicated parsers.

Python Text to numbers and dates

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

strict=False turns bad values into null instead of raising, so you can count the failures afterwards with null_count(). Dates are covered in Date and Time Expressions.

When you can express a text rule as one of these built-in methods, do it. They run in compiled code on the whole column. Calling map_elements with Python's string methods is far slower.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the default interpretation of the pattern in str.contains?

    contains uses regex unless you pass literal=True. Escape special characters or use literal=True for exact matching.
  2. 2. Which returns a list column of every number found in a string?

    extract_all returns all non-overlapping matches as a list. extract returns just one capture group of the first match.
  3. 3. What does str.split('/') produce?

    split returns a List(String) column. split_exact returns a struct, which you can unnest into columns.
  4. 4. How do you convert bad text like 'oops' to null while casting to an integer?

    strict=False replaces unconvertible values with null instead of raising an error.

Summary

  • Text methods live in .str and operate on whole columns; null stays null.
  • Clean with strip_chars, to_lowercase, to_titlecase; search with contains, starts_with, find.
  • Pull out and change parts with extract, extract_all, replace, replace_all; break up with split and split_exact.
  • Build text with concat_str and format; parse with cast(..., strict=False) and to_date.

Next: Date and Time Expressions