C# Reference Β· Strings and Text
Unicode normalization in C#
Reference for C# string.Normalize() and IsNormalized(): NFC, NFD, NFKC and NFKD, composed and decomposed text, comparing accents, and removing diacritics.
Unicode can write the same visible text in more than one way. The letter Γ© can be one character (U+00E9) or an e followed by a combining acute accent (U+0065 U+0301). The two look identical on screen but are different strings: they have different lengths and == says they are not equal.
Normalization converts text to a standard form so that equivalent text becomes identical. In .NET you use string.Normalize() and string.IsNormalized(), with a NormalizationForm. Do it before comparing, hashing or storing text from users or files, when equivalence matters.
using System.Text; string composed = "Γ©"; string decomposed = "eΜ"; bool equal = composed == decomposed; // false bool afterNormalize = composed == decomposed.Normalize(NormalizationForm.FormC); // true on a normal machine
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input Β· what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input Β· what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.
Syntax
public string Normalize() public string Normalize(NormalizationForm normalizationForm) public bool IsNormalized() public bool IsNormalized(NormalizationForm normalizationForm) public enum NormalizationForm { FormC = 1, FormD = 2, FormKC = 5, FormKD = 6 }
| Form | Meaning |
|---|---|
FormC |
Canonical composition: combine base letter and marks into single characters where possible. The usual choice for storage and comparison. |
FormD |
Canonical decomposition: split into base letters and combining marks. Useful for stripping accents. |
FormKC |
Compatibility composition: also replaces look-alike characters (such as the ligature ο¬ or full-width letters) with plain equivalents. |
FormKD |
Compatibility decomposition. |
Returns: a new normalized string (or the same instance if it already was). IsNormalized returns a bool.
Errors / undefined behavior: ArgumentException if the string contains an invalid code point sequence such as a lone surrogate. Behavior depends on the platform's normalization data: with full ICU or NLS data, normalization works as the Unicode standard says; in invariant globalization mode, non-ASCII text is returned as it is, IsNormalized reports false for it, and the invalid-sequence check is skipped.
Usage notes
- Equality of accented text.
"Γ©" == "eΜ"is false. Normalize both sides to the same form (usuallyFormC) before comparing, or compare with a culture-sensitive comparison that treats canonical equivalents as equal. Ordinal comparison never does. - Where to normalize. Normalize at the boundary: when reading user input, file names from different operating systems (macOS tends to produce decomposed names), and before storing keys or usernames.
- Compatibility forms lose information.
FormKC/FormKDturn"β "(circled one) into"1", which is good for search keys and bad for text you must reproduce. - Removing accents is done by decomposing, dropping non-spacing marks, and composing again, as in the example. It does not handle every letter:
ΓΈ(o with stroke) has no decomposition. - Not case folding. Normalization does not change case. Combine with
ToUpperInvariant()for case-insensitive keys; seeToUpper()andToLower(). - Not a security filter on its own. Normalize before validating, not after, or an attacker can slip in characters that become dangerous after normalization.
- Length changes. After
FormDa string can be longer; afterFormCshorter. Do not cache lengths across the call. - Runner limit. Our server runs with invariant globalization, where
Normalize()is effectively a no-op for non-ASCII text. Read the examples above as showing the API; run them locally with a normal .NET install to see the conversions.