Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 38 · Flags and Engines

Unicode and the v Flag

Match text in any language with Unicode regular expressions. Learn code points versus code units, property escapes like \p{L}, case folding, and set operations with the v flag.

Text is not just ASCII. Names contain accents, messages contain emoji, and users type in Arabic, Hindi or Japanese. A pattern like [a-z]+ quietly fails on all of them. Unicode-aware regex gives you character properties instead of hand-written ranges, so one pattern works for every script.

This lesson uses JavaScript, where Unicode mode is opt-in through the u or v flag. Most other modern engines (Python 3, Java, .NET, Rust) are Unicode-aware by default but offer similar property syntax. See regex flavors and engines for the comparison.

Code units versus code points

JavaScript strings are sequences of UTF-16 code units. Characters outside the Basic Multilingual Plane, such as most emoji, take two units (a surrogate pair). Without a Unicode flag, a regex treats each unit as a separate character.

Node.js Surrogate pairs without and with the u flag

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The u flag also makes the engine stricter. An unknown escape like \_ or \- outside a class becomes a syntax error, which is helpful for catching typos.

Property escapes

With u or v, \p{Name} matches one character that has a given Unicode property, and \P{Name} matches one that does not. The most useful ones:

Escape Matches
\p{L} Any letter, in any script
\p{Lu}, \p{Ll} Uppercase or lowercase letter
\p{N}, \p{Nd} Any number, or decimal digit only
\p{P} Punctuation
\p{Sc} Currency symbols
\p{Emoji} Characters with the Emoji property (includes digits, #, *)
\p{Emoji_Presentation} Characters shown as emoji by default
\p{Script=Greek} or \p{sc=Grek} Characters in the Greek script
\p{Script_Extensions=Han} Characters used with the Han script
Node.js Letters in any language

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Notice that \w is still ASCII-only even with u: it means [A-Za-z0-9_]. Use [\p{L}\p{N}_] when you need a Unicode-aware word character.

Case folding

With i alone, JavaScript uses simple case mapping. With iu (or iv) it uses Unicode simple case folding, so ſ (long s) and K (Kelvin sign) match s and k. That is correct for linguistic matching but can surprise security checks, so be deliberate.

JavaScript
/straße/i.test("STRASSE");  // false: ß does not fold to ss in simple folding
/K/i.test("k");        // false without u
/K/iu.test("k");       // true: Kelvin sign folds to k

Combining marks and grapheme clusters

What a user sees as one character may be several code points: é can be e plus a combining accent, and a family emoji joins several people with zero-width joiners. A regex matches code points, not what the eye sees as one symbol. To count visible characters, use Intl.Segmenter, not a regex.

Node.js One symbol, many code points

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

A common trick: normalize text with str.normalize("NFD") and strip \p{M} to remove accents for searching.

The v flag: set notation

The v flag (Node 20+, all current browsers) upgrades u with features inside character classes: nested classes, subtraction --, intersection &&, and string literals with \q{...}. It also gives \p{RGI_Emoji} for whole emoji sequences.

Node.js Set operations with v

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

With v, some punctuation inside classes must be escaped (( ) [ ] { } / - \ | and double punctuators like &&), because they are now syntax. Use u when you need to support older runtimes; use v when you want sets.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. Why does /^.$/ fail on the string "😀" without the u flag?

    Without u the engine works on UTF-16 code units, and an emoji outside the basic plane takes two of them.
  2. 2. Which pattern matches a word made of letters from any script?

    \p{L} is the Unicode letter property. \w is ASCII only, even in Unicode mode.
  3. 3. What does [\p{L}--\p{Script=Latin}] do with the v flag?

    -- is set subtraction in v mode, so the class holds all letters minus the Latin ones.
  4. 4. How should you count the symbols a user sees in a string with emoji sequences?

    Grapheme clusters can contain several code points, which regexes and length do not treat as one unit.

Summary

  • Without u or v, JavaScript regexes work on UTF-16 code units, which splits emoji.
  • \p{...} property escapes match by category or script and replace hand-written ranges.
  • \w, \d and \b stay ASCII-based; build Unicode-aware versions from \p{L} and \p{N}.
  • iu uses Unicode case folding, which can match unexpected characters.
  • The v flag adds set subtraction, intersection, nested classes and \p{RGI_Emoji}.
  • Use normalize() and Intl.Segmenter for combining marks and visible symbols.

Next: Regex Flavors and Engines