Skip to content
Tutorials Center
My learning

Topics

Lesson 107 of 146 Β· Standard Library and Foundation

Characters, Unicode and Grapheme Clusters

Go inside Swift strings - Character vs Unicode scalar vs UTF-8 and UTF-16 views, grapheme clusters, canonical equivalence, emoji sequences and decoding bytes safely.

In most languages a "character" is a number that fits in 16 bits or 32 bits, and a string is an array of them. Swift makes a different promise: a Character is what a human reader sees as one symbol, no matter how many numbers it takes to store it. That is why "πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦".count is 1 in Swift and a surprising 11 or 7 in many other languages.

Strings introduced the idea and Text Processing used it for everyday jobs. This lesson goes one level deeper: the four views of a string, what a grapheme cluster is, why two different byte sequences can be the same string, and how to turn raw bytes into text without crashing. You will not need this for every program, but when you handle names, emoji, file names or network data, it saves hours of debugging.

Four ways to look at the same text

A Swift String is stored as UTF-8 bytes. Four views expose that storage at different levels:

flowchart TD
    S["String"] --> C["Characters<br/>(grapheme clusters)"]
    S --> U["unicodeScalars<br/>(Unicode code points)"]
    S --> U16["utf16<br/>(16-bit code units)"]
    S --> U8["utf8<br/>(bytes)"]

One string, four views. Each view is a collection with its own element type and its own count.

Swift The same text through four views

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

All four counts agree on caf (three ASCII letters), and they only differ on the Γ©. Here Γ© is a single scalar, U+00E9, so Characters, scalars and UTF-16 units all say 4. In UTF-8 that scalar needs two bytes (195, 169), so the byte count is 5.

Which view should you use?

  • Character for anything a user sees: counting letters, reversing text, truncating a label.
  • unicodeScalars when you need the actual code points, for example to classify them.
  • utf8 for sizes in bytes, hashing, file formats and network protocols.
  • utf16 only to talk to APIs that count in UTF-16 units, such as Foundation's NSString.

What is a grapheme cluster?

Unicode assigns a number (a scalar) to every symbol, accent and joiner. Some things you see as one symbol are built from several scalars. A grapheme cluster is the unit that Unicode's text-segmentation rules say a reader perceives as one character. Swift's Character is exactly one grapheme cluster.

Swift One Character, many scalars

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

Read the output line by line:

  • e followed by U+0301 (a combining accent) draws as Γ© and counts as one Character built from two scalars.
  • The flag is two regional indicator letters (J and P). Swift pairs them into one flag.
  • The skin tone is a modifier scalar glued to the hand.
  • The family is four emoji separated by U+200D, the zero-width joiner, which asks the renderer to draw them as one picture.

Because a Character can be arbitrarily large, the old assumption "one character takes a fixed number of bytes" is gone. Operations like count and index(_:offsetBy:) have to walk the text and apply the segmentation rules. That is the reason String is not randomly indexable by integers (see Strings).

Grapheme rules are part of the Unicode standard and are updated over time. A newer Swift toolchain or operating system can group a new emoji sequence as one Character where an older one counted two. Do not hard-code counts of emoji in tests that must run on several platforms.

Two strings, one meaning: canonical equivalence

Look at the two ways to write Γ© in the previous example. One is a single scalar, the other is e plus a combining mark. They look identical and are defined by Unicode as canonically equivalent. Swift's == follows that definition.

Swift == compares meaning, not bytes

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

The strings compare equal and both have four Characters, yet their bytes are different. The Set holds one element, because Hashable follows ==. A Set or Dictionary therefore treats both forms as one key, which is what you want when a user searches for "cafΓ©" and the file name was typed on a different keyboard. Sorting compares the canonical forms scalar by scalar, so Γ© sorts after f (U+00E9 is larger than U+0066), which is why locale-aware sorting needs localizedStandardCompare from Foundation.

The flip side: do not compare utf8 arrays to test whether two strings are the same text. If you need byte-for-byte identity (for example, a checksum of exact file content), compare the byte views on purpose.

If you must store or send a string in one agreed form, normalize it first. Foundation offers precomposedStringWithCanonicalMapping (NFC) and decomposedStringWithCanonicalMapping (NFD) on Apple platforms and in swift-corelibs-foundation. Standard-library == already treats both forms as equal, so you rarely need it for comparison.

Looking inside a Character

Character has useful properties that work on the whole cluster:

Swift Character and scalar properties

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

Note that isNumber and wholeNumberValue understand other scripts: the Arabic-Indic digit Ω£ is a number with value 3. If your program parses only ASCII digits, check isASCII as well, or use Int("..."), which accepts only ASCII digits and a sign.

The properties of a Unicode.Scalar expose the full Unicode Character Database: general category, case mapping, emoji flags and more.

Indices, offsets and safe slicing

String indices point to a grapheme boundary. You get them from the string, never from an integer. When you must convert, go through the view that matches the units you were given:

Swift Moving between views

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

The same String.Index works in all views, but the distance it represents depends on the view you measure it in. The rocket is the 7th Character (offset 6), starts at byte offset 7 because Γ― takes two bytes, and sits at UTF-16 offset 6. Mixing up views is the classic cause of cut-off emoji and garbled text.

Turning bytes into text

Bytes from a file or socket are not guaranteed to be valid UTF-8. Swift gives you two policies:

Swift
// Repairs invalid input: bad bytes become U+FFFD
let lenient = String(decoding: bytes, as: UTF8.self)

// Strict decoding is done by hand with the UTF8 codec (see below)
Swift Valid, repaired and rejected UTF-8

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

Use the repairing initializer for display (a log viewer would rather show οΏ½ than refuse a line) and the strict path when corrupt data must be rejected, such as a configuration file or a signature check. See Data and Bytes for more on byte buffers.

Check your understanding

Quick check Β· 5 questions

of 5 answered You scored / 5 Β· perfect!

  1. 1. What does Character represent in Swift?

    A Character is what a reader perceives as a single symbol, so e plus a combining accent, a flag or a family emoji are each one Character.
  2. 2. Which view should you use to measure how many bytes a string takes when written to a file as UTF-8?

    The utf8 view exposes the stored bytes, so its count is the size in bytes. count counts Characters.
  3. 3. "caf\u{E9}" == "cafe\u{301}" evaluates to what, and why?

    Both spell the same text according to Unicode, so == is true and the two strings hash the same, even though their UTF-8 bytes differ.
  4. 4. What does String(decoding: bytes, as: UTF8.self) do with an invalid byte?

    That initializer repairs bad input. To reject invalid data, decode manually or use a failable API.
  5. 5. Why is "πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦".count equal to 1 even though it has seven scalars?

    The four people are joined by U+200D, and the segmentation rules treat the whole sequence as one cluster.

Summary

  • A String has four views: Characters, unicodeScalars, utf16 and utf8, each with its own count.
  • A Character is a grapheme cluster: one perceived symbol, possibly many scalars (accents, flags, skin tones, joined emoji).
  • ==, hashing and ordering use canonical equivalence, so precomposed and decomposed text are equal while their bytes differ.
  • Character and scalar properties (isLetter, wholeNumberValue, properties.isEmoji) understand every script.
  • Measure with the view that matches your unit, and never reuse an offset from one view in another.
  • Decode untrusted bytes deliberately: repair with String(decoding:as:) or reject invalid input.

Next: Memory Layout