Skip to content
Tutorials Center
My learning

Topics

Lesson 139 of 146 ยท Projects

Project: A JSON Parser

Build a JSON parser in Swift from scratch - an enum for JSON values, a recursive descent parser with typed throws, line and column errors, a pretty printer, path queries and round-trip tests.

You already know how to read JSON in Swift: JSONDecoder turns text into your own types in one line. This project goes underneath. You will write the part that turns raw text into a tree of values, which is what every JSON library does first. Along the way you will use almost everything from the language that matters for this kind of program: enums with associated values, recursion, switch with patterns, typed throws, Unicode scalars, key paths, and subscripts of strings by index.

JSON is an ideal project for another reason: it is small. The whole grammar fits on a postcard, yet it contains the three challenges that appear in every parser. You must decide what the data model is, you must write a recursive algorithm for a recursive format, and you must produce useful errors when the input is wrong. This is the same technique as the expression calculator, applied to a bigger grammar.

The grammar in one minute

A JSON value is exactly one of:

  • null, true or false
  • a number such as -12, 3.5 or 6.02e23
  • a string in double quotes with backslash escapes
  • an array: [ values separated by commas ]
  • an object: { string keys, each followed by : and a value, separated by commas }

Arrays and objects contain values, which can be arrays and objects. That recursion is the whole difficulty, and it is the reason the parser is recursive too: one function per kind of value, calling each other.

Step 1: the data model

An enum whose cases are the six kinds of value, with payloads, models JSON exactly. The type is recursive because array holds [JSON]. Unlike the Expr type from the indirect enums lesson, it does not need the indirect keyword: an Array and a Dictionary already keep their elements on the heap, so the enum has a fixed size. You only need indirect when a case holds the enum itself directly.

Next, a serializer that writes a value back as compact text. It must escape quotes, backslashes and control characters in strings, and it prints whole numbers without a trailing .0. Object keys are sorted so that the output is always the same; real JSON does not guarantee any key order, and Swift dictionaries randomize theirs.

Swift Step 1: the JSON enum and a serializer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

JSON: Equatable is synthesized, because every payload type is itself Equatable. That gives you == for free, which the round-trip tests at the end rely on.

Adding a case to the enum later (say, separate integer and double cases) turns every incomplete switch into a compile error. The next example shows what that error looks like, so you recognize it when it saves you.

Swift A non-exhaustive switch (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

The compiler stops with error: switch must be exhaustive and the note add missing case: '.number(_)'. Nothing was forgotten silently: the type system found the hole. Add the missing branch and the program compiles.

Step 2: a recursive descent parser

Recursive descent means: write one method per grammar rule, and let the methods call each other the way the rules refer to each other. parseValue looks at the next character, which is enough in JSON to know what comes next, and hands off to parseArray, parseObject, parseString, parseNumber or a literal check. parseArray calls parseValue for each element, which is where the recursion happens.

Design decisions, in order of importance:

  • Scan Unicode scalars, not Characters. JSON syntax is ASCII, while string contents can be any Unicode. Working with an Array<Unicode.Scalar> gives constant-time indexing, which String deliberately does not. It uses 4 bytes per character, which is fine here; a production parser would scan UTF-8 bytes instead.
  • Typed throws. Every method is declared throws(ParseError). Callers can then catch with no cast and know the full set of failures, as the typed throws lesson describes.
  • Positions. The parser keeps an index. When something is wrong, it converts that index into a line and column by counting newlines, so a person can find the mistake.
  • A depth limit. Input like [[[[[... nested a million times would exhaust the call stack and crash the process. A hostile document must produce an error, not a crash, so the parser counts depth and gives up at 64.
Swift Step 2: the parser, with good and bad input

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

The four good inputs print as compact, sorted JSON, with escapes restored and รฉ decoded to รฉ and the surrogate pair to a single emoji. The seven bad inputs each produce one error:

TEXT
error: unexpected character ']' at line 1, column 7
error: expected ':' after the key at line 1, column 6
error: unexpected character '@' at line 2, column 3
error: unterminated string at line 1, column 14
error: unexpected extra text at line 1, column 2
error: expected 'true' at line 1, column 10
error: nesting is too deep at line 1, column 65

Read parseString once slowly, because it holds most of the subtle logic. It walks scalar by scalar. A plain scalar is appended. A backslash makes it look at the next scalar to decide what to append; the \u case reads four hex digits and, for a high surrogate (D800 to DBFF), requires an immediately following \uDC00 to \uDFFF low surrogate and combines the pair into one scalar above FFFF. That is how ๐Ÿ˜€ becomes a single emoji. A lone surrogate is invalid JSON text in practice, and Unicode.Scalar(code) returns nil for it, which becomes an error rather than a corrupt string.

A few observations from the output. The error for [1, 2,] points at the closing bracket, because after the comma the parser expected another value. In the multi-line input, the line and column count newlines, so the report says line 2. And the 100 opening brackets stop at the depth limit instead of overflowing the stack.

A Double has 53 bits of precision, so numbers beyond 2^53 lose exactness. The input 9007199254740993 becomes the nearest representable double, and the run above prints 9007199254740992.0: the last digit changed, and the serializer's whole-number shortcut applies only below 1e15. Many JSON libraries have this property, because JSON itself does not specify number precision. If you need exact large integers, keep the number's text, or add an integer(Int64) case and choose it when the text has no fraction or exponent.

Typed throws, in practice

Because the whole parser is throws(ParseError), the catch block in a function that calls it would know error is a ParseError directly, with no as?. The compiler also enforces it. Try to throw a different error from a function with a typed signature:

Swift A typed throws mismatch (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

The compiler answers error: thrown expression type 'DiskError' cannot be converted to error type 'ParseError'.

The fix is to convert at the boundary, by mapping the lower-level error into a ParseError case, or to widen the signature to plain throws. Typed throws earns its keep inside self-contained components like this one, where the set of failures is small and closed.

Step 3: pretty printing and queries

A parser is only useful if you can do something with the tree. Two features complete the project. A pretty printer writes the value with indentation, and a query language such as users[1].name looks inside a document.

The query path is itself a recursive structure, so it is a natural fit for an indirect enum. A Query is the root, or a key lookup applied to a smaller query, or an index lookup applied to a smaller query. Evaluating the outermost query evaluates its parent first. Here indirect is required, because key holds a Query directly.

Swift Step 3: pretty printer, queries and tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readLine()), it gets the next line from this box.

The program prints the document with two-space indentation (keys sorted, depth: 3), then answers the eight scripted queries and finishes with the test result:

TEXT
$.name -> "swift-notes"
$.tags[1] -> "enums"
$.authors[5] -> no match
$.dependencies -> []
answered 8 queries
round trips that matched: 300

The query part shows why a recursive type makes recursive code short. evaluate on .index(.key(.root, "authors"), 0) first asks its parent, .key(.root, "authors"), which asks .root. Each level either narrows the document or gives up with nil, and nil flows outward because guard case ...? = parent.evaluate(in: json) fails on nil. The ? in .object(let members)? is an optional pattern, which matches only when the optional holds a value of that case; see pattern matching. Query.parse builds the same structure from text, using Substring to take pieces of the path without copying.

The last block is the most valuable one for a real project. It is a property test: generate hundreds of values from a seeded generator, serialize each one, parse it again, and check that you get the original back, both for compact and pretty output. A fixed seed makes the "random" values the same on every run, so a failure can be reproduced. precondition stops the program with a message on the first mismatch; in this sandbox, builds are optimized, so you would see a crash rather than the message text. Round-trip tests catch the bugs that hand-written examples miss: an escape you forgot, a number format the parser rejects, or a surrogate pair split incorrectly. Note that the serializer writes control characters as \u00XX, which is exactly the kind of detail such a test checks.

Where to take it next

  • Streaming. Parse from bytes as they arrive instead of a whole String. The same recursive structure works with an AsyncSequence of chunks.
  • A real fast path. Scan String.utf8 rather than building an array of scalars, and avoid copying keys and strings until they are needed.
  • Decode into your types. Write a JSONDecoder of your own on top of this tree, using the Decoder protocols from the Codable lesson.
  • Preserve key order. Replace [String: JSON] with an array of key-value pairs when order matters, for example when you rewrite a configuration file and want a minimal diff.
  • Better errors. Report the expected token and print the offending line with a caret under the column.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does enum JSON not need the indirect keyword even though array contains [JSON]?

    indirect is needed only when a case holds the enum value itself inline. An Array is a small fixed-size handle to heap storage, so the type's size is known.
  2. 2. What does recursive descent mean in this parser?

    parseValue dispatches to parseArray and parseObject, which call parseValue again for their elements, mirroring the recursive grammar.
  3. 3. Why does the parser enforce a maximum nesting depth?

    Recursion uses stack space proportional to the depth. Untrusted input must never be able to crash the process, so the parser stops at a fixed depth.
  4. 4. In throws(ParseError), what does the type in parentheses give callers?

    Typed throws makes the set of failures part of the signature. The mismatch example shows the compiler refusing to throw a DiskError from it.
  5. 5. What does the seeded round-trip test check?

    A fixed seed makes the random inputs reproducible. Equality of the re-parsed value with the original shows that serializer and parser agree, including escapes, numbers and Unicode.

Summary

  • Model JSON as an enum with six cases; arrays and dictionaries make it recursive without needing indirect.
  • A recursive descent parser has one method per grammar rule, scans Unicode scalars by index, and uses a depth limit so hostile input cannot crash it.
  • Typed throws (throws(ParseError)) makes the failure type part of the signature, and positions computed from the index give line and column in error messages.
  • Strings need escapes both ways, including \u sequences and surrogate pairs; numbers stored as Double lose exactness beyond 2^53.
  • A recursive indirect enum Query shows where indirect is required, and optional patterns (case .object(let m)?) make chained lookups short.
  • Seeded property tests that round-trip generated values find bugs that hand-picked examples miss.

Next: Swift Cheat Sheet