Skip to content
Tutorials Center
My learning

Topics

Lesson 82 of 147 ยท Standard Library

Regular Expressions

Learn Go regular expressions with the regexp package - RE2 syntax, MatchString, Find and FindAll, capture groups, named groups, ReplaceAllString, flags, compiling once and what RE2 cannot do.

A regular expression (regex) is a compact pattern that describes a set of strings. You use one to check that text has a certain shape (is this an email-like string?), to pull out pieces (the year, month and day of a date) or to rewrite text (mask every digit). Go's regexp package implements these operations, and it makes an unusual promise: matching time grows linearly with the size of the input, no matter how nasty the pattern is.

That promise comes from the engine behind the package, RE2. This lesson shows the syntax, the search and replace functions, capture groups, and the limits you accept in exchange for that safety.

Matching: does it fit?

Before anything else, a program must compile a pattern into a *regexp.Regexp. regexp.MustCompile panics on an invalid pattern, which is right for patterns written in your source code, while regexp.Compile returns an error, which is right for patterns that come from users. Use raw strings (backticks) for patterns, so you don't have to double every backslash.

Go MatchString and MustCompile

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

^ and $ anchor the pattern to the start and end of the text. Without them, MatchString reports a match anywhere in the string, so \d{5} alone would also match inside "abc12345xyz".

Compile a pattern once and reuse it: package-level var re = regexp.MustCompile(...) is the standard idiom. Compiling is far more expensive than matching, and a *Regexp is safe for use by many goroutines at the same time. Avoid regexp.MatchString and MustCompile inside loops or request handlers.

A quick tour of the syntax

Go uses the RE2 syntax, which will look familiar if you've used Perl, Python or JavaScript regexes. The most useful pieces:

Pattern Meaning
. any character except newline
\d, \w, \s digit, word character ([0-9A-Za-z_]), whitespace
\D, \W, \S the opposites
[abc], [^abc], [a-z] character class, negated class, range
[[:alpha:]], \pL POSIX class, Unicode letter
^, $ start and end of text (or of line with (?m))
\b ASCII word boundary
*, +, ? zero or more, one or more, zero or one
, {n,}, {n,m} exact and ranged repetition
*?, +?, ?? the same but non-greedy (take as little as possible)
a|b alternation
(...), (?:...) capturing group, non-capturing group
(?P<name>...) or (?<name>...) named capturing group

Quantifiers are greedy by default: they match as much as they can. Add ? to make them lazy:

Go Greedy versus lazy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

(Parsing real HTML with regexes is a bad idea; this only illustrates the quantifiers.)

Find: the family of search methods

The Find methods follow a naming scheme. Pick the pieces you need:

  • Return type: Find returns the matched []byte, FindString a string, and the Index variants []int positions. Methods with Submatch also return the capture groups.
  • How many: a method without All returns only the first match; All methods return every non-overlapping match, limited by an n argument (-1 means all).

So FindAllStringSubmatch finds all matches, as strings, with their submatches.

Go Find, FindAll and positions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

When nothing matches, FindString returns "" (indistinguishable from an empty match, so use MatchString or FindStringIndex if the difference matters) and the All functions return nil. Positions are byte offsets, as with strings.

Capture groups

Parentheses create groups. FindStringSubmatch returns a slice whose element 0 is the whole match and whose other elements are the groups, in the order of their opening parentheses. A group that didn't take part in the match is an empty string.

Giving groups names makes patterns readable and robust against reordering. SubexpNames lists the names, and SubexpIndex converts a name to an index:

Go Capture and named groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Always check that the result has the length you expect (m == nil means no match) before indexing into it; indexing a nil slice panics.

Replacing text

ReplaceAllString substitutes matches with a template. In the template, $1 or ${1} insert the first group and ${name} inserts a named group. Write $$ for a literal dollar sign. Because $1x would be read as a group called 1x, prefer the braced form when text follows.

For literal replacement text without any $ handling use ReplaceAllLiteralString, and for full control use ReplaceAllStringFunc, which calls your function with each match:

Go Replace, expand and transform

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

ExpandString appends the expanded template to a byte slice and returns it, so it is typically used with the index slices from the ...Index methods. For most replacement jobs ReplaceAllString is simpler.

Flags

Flags change how the whole pattern, or a part of it, behaves. They are written as (?flags) at the start, or (?flags:...) around a section:

Flag Effect
i case-insensitive
m multi-line: ^ and $ match at line starts and ends
s . also matches a newline
U swap greedy and lazy behavior
Go Flags in action

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

regexp.QuoteMeta escapes every special character, so text typed by a user is matched literally. For plain substring checks, though, strings.Contains is simpler and faster than any regex.

What RE2 guarantees, and what it gives up

Many regex engines (Perl, Python, Java, JavaScript) use backtracking: they try one path through the pattern and, on failure, return to earlier choices and try another. For nested quantifiers such as (a+)+$ on the text aaaaaaaaaaaaaaaaaaaaaaaa!, the number of paths explodes exponentially. This is catastrophic backtracking, and attackers use it for ReDoS (regular-expression denial of service) attacks.

RE2 instead simulates all paths at once, in time proportional to the pattern size times the input size. The same pattern that freezes other languages finishes immediately in Go:

Go No catastrophic backtracking

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The price is a smaller feature set. RE2 does not support:

  • lookahead and lookbehind ((?=...), (?!...), (?<=...), (?<!...)),
  • backreferences (\1 inside a pattern to repeat an earlier group),
  • possessive quantifiers and atomic groups.

Trying to compile them returns an error at once:

Go Unsupported syntax is rejected

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

When you need these features, restructure the problem: match a wider pattern and test the surrounding text in code, run two simple expressions, or write a small parser. Regexes are best for flat, regular text; nested structures such as HTML, JSON or balanced parentheses need a real parser.

Practical tips

  • Use \b only for ASCII words; for other scripts match with \p{L} classes.
  • By default Find is leftmost-first like Perl. regexp.CompilePOSIX and re.Longest() switch to leftmost-longest, which changes results for alternations like a|ab.
  • Matching is on UTF-8 text; . matches one rune, not one byte.
  • Patterns from untrusted users are safe from ReDoS in Go, but a very long pattern still costs time and memory. Limit its length.
  • For a simple split, trim or prefix check, the strings package is faster and clearer.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the pattern \d+ with FindString("abc 123 def 45") return?

    FindString returns only the first (leftmost) match. FindAllString would return both numbers.
  2. 2. Why is var re = regexp.MustCompile(...) placed at package level?

    Compile once, match many times. A *Regexp can be shared by many goroutines.
  3. 3. What happens when you compile the pattern (\w)\1 in Go?

    RE2 trades backreferences and lookarounds for guaranteed linear-time matching.
  4. 4. With the pattern (\w+)@(\w+), what does ReplaceAllString("ann@home", "${2}:${1}") return?

    ${1} is the first group (ann) and ${2} the second (home), so the template swaps them.
  5. 5. Without any flags, what does ^\w+ find in a two-line text, "one" then a newline then "two", with FindAllString?

    Without (?m), ^ only matches at the very start of the text. With (?m) it would also match after the newline.

Summary

  • Compile with regexp.MustCompile (fixed patterns) or regexp.Compile (patterns from input), once, usually in a package-level variable.
  • MatchString tests, Find... extracts (All for every match, Submatch for groups, Index for positions), ReplaceAll... rewrites, Split cuts.
  • Capture groups are numbered by opening parenthesis; name them with (?P<name>...) and use SubexpIndex.
  • In templates, use ${1} or ${name}; ReplaceAllLiteralString and ReplaceAllStringFunc avoid or replace template rules.
  • Flags (?i), (?m), (?s) and (?U) change matching; QuoteMeta escapes literal text.
  • RE2 guarantees linear-time matching with no catastrophic backtracking, but has no lookaround or backreferences.

Next: Templates: text/template and html/template turns data into text and web pages.