Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 38 ยท Flags and Engines

Regex Flavors and Engines

Understand why the same regular expression behaves differently across languages. Compare backtracking and automaton engines, the PCRE, JavaScript, Python, Java and RE2 flavors, and portable syntax.

"Regex" is not one language. It is a family of dialects called flavors, and each language or tool picks its own. The pattern \d+ works everywhere, but (?<name>...), \A, possessive quantifiers or \p{L} are accepted by some flavors and rejected by others. Knowing which flavor you are targeting saves hours of "it works in my tester but not in my code".

There are two separate questions: what syntax does the flavor accept, and which algorithm runs the match. The algorithm decides not just features but also speed and safety.

Two families of engines

A backtracking engine tries to match the pattern from left to right. When a choice fails, it goes back and tries the next alternative. This is simple, supports powerful features such as backreferences and lookaround, and gives intuitive rules: the first alternative that works wins. The cost is that bad patterns can take exponential time (ReDoS).

An automaton-based engine (DFA, NFA simulation) compiles the pattern into a state machine and walks the input once. Matching time is proportional to the input length, no matter the pattern, which makes it safe for untrusted patterns. The price: no backreferences and usually no arbitrary lookaround.

flowchart LR
  P["Pattern"] --> C{"Engine type"}
  C -->|"Backtracking"| B["Tries alternatives, goes back on failure<br/>Backreferences, lookaround<br/>Worst case exponential"]
  C -->|"Automaton"| A["Builds a state machine<br/>One pass over the input<br/>Linear time, fewer features"]

Which engine a language uses decides what is possible and what can go wrong.

Who uses what

Flavor Engine Notes
JavaScript (V8, SpiderMonkey, JavaScriptCore) Backtracking Lookbehind, named groups, v flag. No atomic groups, no possessive quantifiers. V8 also ships an optional linear engine (--enable-experimental-regexp-engine).
PCRE and PCRE2 (PHP, R, many tools) Backtracking The richest feature set: atomic groups, possessive quantifiers, recursion, conditionals, \K.
Python re Backtracking Close to PCRE; since 3.11 has atomic groups and possessive quantifiers. Third-party regex module adds more.
Java java.util.regex Backtracking Possessive quantifiers, atomic groups, \p{L}.
.NET Backtracking Balancing groups; has RegexOptions.NonBacktracking and match timeouts.
Ruby (Onigmo) Backtracking Many PCRE features plus absence operators.
Go regexp, Rust regex, RE2 Automaton Linear time; no backreferences, no lookaround.
POSIX BRE and ERE (grep, sed, awk) Automaton or backtracking Older, smaller syntax; leftmost-longest semantics in strict POSIX.

What differs between flavors

Syntax differences cluster in a few areas:

  • Anchors for the whole string. \A and \z exist in PCRE, Python (\A, \Z), Java, Ruby. JavaScript has only ^ and $ (plus the m flag).
  • Named groups. (?<name>x) in JavaScript, Java, .NET, PCRE, Ruby, not Python, which only accepts (?P<name>x). Go and Rust accept both (?P<name>x) and (?<name>x).
  • Lookbehind. Supported in JavaScript, PCRE, Java, Python (fixed width), .NET (any width). Not in RE2, Go, Rust.
  • Inline flags. (?i) at the start works in PCRE, Python, Java, Go, Rust. JavaScript has no (?i) but has scoped modifiers (?i:...) in current engines.
  • Unicode. Python 3, Java and Rust treat strings as Unicode by default; JavaScript needs u or v; PCRE needs the u modifier (PHP) or (*UTF).
  • Replacement syntax. $1 or $& in JavaScript, \1 or \g<1> in Python, $1 in Java and PHP (\1 also works in PHP).
  • Dot and newlines. In most flavors . skips \n; the dotall flag is s (JavaScript, PCRE, Rust, Go), re.S (Python), DOTALL (Java).
Node.js JavaScript features that depend on the engine

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The same ideas written for PCRE (the engine behind PHP) can use possessive quantifiers and atomic groups to forbid backtracking:

PHP PCRE: possessive quantifiers and atomic groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with fgets(STDIN)), it gets the next line from this box.

Writing portable patterns

If a pattern has to work in several languages (a shared config, a rules file, a pattern stored in a database), stay inside the common core:

  1. Literals, ., classes [...], ranges, \d \w \s (and know that their Unicode meaning differs).
  2. Groups ( ), alternation |, quantifiers * + ? {n,m} and lazy *?.
  3. Anchors ^ and $, and \b for word boundaries.
  4. Avoid: lookbehind, named groups, backreferences, atomic groups, inline flags, \Z.

When the pattern comes from users, prefer a linear engine (RE2, Go, Rust) or a library with timeouts. The performance lessons (backtracking, hardening) explain why.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which of these features is typically NOT available in a linear-time engine such as RE2?

    Backreferences cannot be handled by a plain state machine, so RE2, Go and Rust do not support them.
  2. 2. What is the worst-case danger of a backtracking engine?

    Nested or overlapping quantifiers can force the engine to try an exponential number of paths.
  3. 3. In PCRE, what does /a++a/ do on the string aaaa?

    Possessive quantifiers never backtrack, so a++ takes all four a characters and the final a has nothing left.
  4. 4. Which anchor matches only at the very end of the string in PCRE, even before a trailing newline?

    $ also matches before a final newline by default, but \z means the absolute end of input.

Summary

  • A flavor is a syntax dialect; an engine is the algorithm. Both matter.
  • Backtracking engines (JavaScript, PCRE, Python, Java, .NET, Ruby) are powerful but can be exponential.
  • Automaton engines (RE2, Go, Rust) guarantee linear time but drop backreferences and lookaround.
  • Differences show up in anchors, named groups, lookbehind, inline flags, Unicode defaults and replacement syntax.
  • For portable patterns use only the common core; for untrusted patterns prefer a linear engine.

Next: Sticky Matching and Match Indices