Skip to content
Tutorials Center
My learning

Topics

Lesson 58 of 149 ยท Modules and the Standard Library

Regular Expressions

Learn regular expressions in Python with the re module. Write patterns, match and search text, capture groups, find all matches, replace text and compile patterns.

String methods like find(), split() and replace() are great when you know the exact text you're looking for. But what if you need "any date", "every email address" or "a word followed by a number"? That's the job of regular expressions (regex): small patterns that describe the shape of text.

Python supports them through the built-in re module. Regex syntax is compact and can look cryptic at first, but a dozen symbols cover most real-world needs. This lesson builds them up one at a time, then shows the functions that search, extract, split and replace.

Your first pattern

A pattern made of ordinary characters matches those characters literally. re.search() scans a string and returns a match object for the first hit, or None:

Python re.search

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Always write patterns as raw strings (r"..."). Regex uses lots of backslashes, and the r prefix stops Python from interpreting them first. Without it, "\b" would be a backspace character instead of the regex word boundary.

The building blocks

Character classes

These match one character from a set:

Pattern Matches
. any character except a newline
\d / \D a digit / anything but a digit
\w / \W a word character (letter, digit, _) / anything else
\s / \S whitespace (space, tab, newline) / anything else
[aeiou] one of the listed characters
[a-z0-9] a character in a range
[^0-9] any character not listed

To match a special character literally, escape it with a backslash: \. matches a real dot, \$ a dollar sign. re.escape(text) escapes a whole string for you, which is essential when a pattern includes user input.

Quantifiers

Quantifiers say how many times the previous item repeats:

Pattern Meaning
* 0 or more
+ 1 or more
? 0 or 1 (optional)
exactly 3
{2,4} 2 to 4
{2,} 2 or more

Anchors and alternatives

Pattern Meaning
^ / $ start / end of the string (or line, with re.MULTILINE)
\b a word boundary
a|b a or b
( ) a group

Put the pieces together and experiment. Each line below tests a pattern against several strings:

Python Pattern playground

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Finding every match

re.findall() returns all matches as a list of strings. re.finditer() yields match objects one by one, which also tell you where each match is:

Python findall and finditer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(?:...) is a non-capturing group: it groups the cents part so ? makes it optional, without capturing it separately. That matters because of how findall() treats groups, as you'll see next.

Groups: capturing parts of a match

Parentheses capture the text they match, so you can pull a match apart. group(0) (or just group()) is the whole match; group(1), group(2) and so on are the parenthesized parts. Named groups, written (?P<name>...), make the code self-explanatory:

Python Capturing groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The findall() rule: no groups returns whole matches; one group returns a list of that group; two or more groups return a list of tuples.

match, search and fullmatch

Three functions check one string in slightly different ways:

  • re.search(p, s) finds the pattern anywhere.
  • re.match(p, s) only matches at the start.
  • re.fullmatch(p, s) requires the whole string to match.

For validating input, fullmatch() is usually what you want, because it doesn't accept extra text before or after:

Python Validating with fullmatch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Replacing and splitting

re.sub() replaces every match. The replacement can refer to groups with \1 or \g<name>, or it can be a function that receives each match and returns the new text:

Python re.sub and re.split

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The masking example uses a lookahead, (?=...): it checks that four digits follow without consuming them, so only earlier digits are replaced. re.subn() works like sub() but also returns how many replacements were made.

Flags and compiled patterns

Flags change how a pattern behaves. Pass them as the flags argument, combined with |:

  • re.IGNORECASE (re.I): ignore letter case.
  • re.MULTILINE (re.M): ^ and $ match at every line, not just the string's ends.
  • re.DOTALL (re.S): . also matches newlines.
  • re.VERBOSE (re.X): ignore whitespace and allow # comments inside the pattern, so complex patterns stay readable.

When you use one pattern many times, re.compile() turns it into a pattern object with the same methods (search, findall, sub and so on):

Python compile with VERBOSE and flags

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python caches recently used patterns internally, so compiling isn't required for speed in small scripts. Its real benefit is a named, documented, reusable pattern.

Don't over-validate emails. The pattern above is fine for finding addresses in text, but the full rules for valid email addresses are famously complex. To really validate one, send a confirmation message.

Greedy vs lazy

Quantifiers are greedy: they match as much as possible. Add ? after one to make it lazy, matching as little as possible. This difference bites when extracting text between delimiters:

Python Greedy and lazy quantifiers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Regex is perfect for small, regular patterns, but don't use it to parse real HTML, JSON or programming languages. Use a proper parser such as html.parser or the json module.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why are regex patterns usually written as raw strings like r"\d+"?

    Without r, sequences like \b are turned into special characters by Python before the regex engine ever sees them.
  2. 2. What does re.findall(r"(\w+)@(\w+)", "a@b c@d") return?

    With two or more groups, findall() returns a list of tuples, one tuple of groups per match.
  3. 3. Which function confirms that an entire input string is a 5-digit code?

    search and match would also accept "123456" or "12345abc". fullmatch requires the whole string to match.
  4. 4. What does re.sub(r"(\w+) (\w+)", r"\2 \1", "hello world") return?

    \1 and \2 in the replacement refer to the captured groups, so the two words swap places.
  5. 5. In <.+?>, what does the ? after + do?

    A ? after a quantifier switches it from greedy to lazy, so each tag is matched separately.

Summary

  • Regular expressions describe the shape of text; Python's re module runs them. Write patterns as raw strings.
  • Core syntax: classes (\d, \w, \s, [...]), quantifiers (*, +, ?, {m,n}), anchors (^, $, \b), alternatives (|) and groups (( ), (?P<name>...), (?:...)).
  • search() finds the first match anywhere, match() checks the start, fullmatch() checks the whole string; findall() and finditer() get every match.
  • sub() replaces matches (with group references or a function); split() splits on a pattern.
  • re.compile() with flags like IGNORECASE, MULTILINE and VERBOSE creates reusable, readable patterns. Quantifiers are greedy unless followed by ?.

Next: Paths with pathlib, where you'll work with files and folders the modern way.