Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 61 ยท Parsing

Syntax Errors and Error Recovery

Learn how parsers report syntax errors and keep going: panic mode, synchronization tokens, phrase-level recovery, and how to write helpful diagnostics.

Most of the programs a compiler sees are wrong. A programmer types a line, runs the build, and expects to be told what is broken, where, and ideally how to fix it. Programs under construction are full of missing semicolons, unbalanced parentheses and half-typed names. A parser that stops at the first mistake forces a slow cycle of fix one, rebuild, fix the next. A good parser reports several real problems in one pass, and does not invent fake ones.

That makes error handling a feature, not an afterthought. In this lesson you learn what a parser should do when it meets a token it cannot accept: how to report it, how to resynchronize so parsing can continue, and how to avoid the avalanche of confusing follow-up messages known as cascading errors. All examples use the recursive descent style from writing a recursive descent parser, but the ideas apply equally to LL and LR parsers.

What a parser can detect

A parser only detects errors that break the grammar: a token that cannot follow the tokens before it. A property known as the viable prefix property says that LL and LR parsers notice an error as soon as the prefix read so far cannot be extended to any valid program. Given let a = 4 + ;, the parser knows something is wrong exactly at ;, not later.

This tells you where the error is detected, which is not always where the mistake was made. If you forget a closing parenthesis, the parser might only notice many tokens later. A good message points at the place where the parser got confused, and often also mentions the earlier construct that is still open.

Errors fall into several classes. The lexer catches bad characters and unterminated strings. The parser catches syntax errors. Later phases catch type errors and undeclared names. This lesson is about the second class.

Goals of error recovery

A recovery strategy should:

  1. Report the error clearly with a position and an understandable message.
  2. Recover quickly so that the rest of the file can still be checked.
  3. Avoid cascades: one real mistake should produce one message.
  4. Add little cost to parsing programs that are correct.

Notice what is not a goal: producing correct code. Once any syntax error is reported, the compiler normally refuses to generate output. Recovery is only for finding more errors in the same run.

Strategy 1: panic mode

Panic-mode recovery is the simplest strategy and the most common one. When the parser finds an error, it discards input tokens until it reaches a synchronizing token, one that marks a place where parsing can safely restart, then continues as normal.

Typical synchronizing tokens are statement terminators (;), closing braces (}), and keywords that begin statements (let, if, while). Intuitively, "if my statement is broken, throw away the rest of it and pick up at the next one."

flowchart TD
    A["Parse statement"] --> B{"Token acceptable?"}
    B -- "yes" --> C["Continue the statement"]
    B -- "no" --> D["Record the error"]
    D --> E["Skip tokens until a synchronizing token"]
    E --> F["Resume at the next statement"]
    C --> F

Panic mode: an error costs the rest of one statement, then parsing resumes.

The implementation in a recursive descent parser needs only a try/except around each statement. The parse functions raise an exception at the problem; the statement loop catches it, stores the message and calls synchronize(). The example below also contains a deliberately naive recovery (skip exactly one token and try again) so you can see why picking good synchronization points matters.

Python Panic mode versus naive skipping

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both strategies see the same five-line program, but naive skipping produces seven messages, three of them noise: after the first error it resumes in the middle of a statement and complains about leftovers that were never real mistakes. Panic mode synchronizes at ; and keywords, reports four errors, and each one points at a real problem. Notice that the caret display is generated from the position stored in the exception, a technique you also need in lexical error reporting.

Two details are easy to get wrong:

  • Always make progress. If the offending token is a statement keyword, synchronize() returns without consuming it. That is only safe because the failing rule has already consumed that keyword. A recovery routine that stops on the very token that just failed would loop forever, so make sure at least one token is consumed per error.
  • Choose tokens carefully. Too few synchronizing tokens throws away large parts of the input and misses later errors. Too many resumes in the middle of broken code and creates cascades. A useful rule: use FOLLOW sets, the tokens that can legally come after the current construct, as the starting point. You computed them in FIRST and FOLLOW sets.

Strategy 2: phrase-level recovery

Phrase-level recovery repairs the input locally so the parser can go on from exactly where it is. Typical repairs insert a token the parser expected (a missing ; or )), delete a stray one, or replace one with another.

This works well for mistakes that are almost always the same: forgetting a terminator. The parser can see that the next token is something that would be perfectly legal at the start of a new statement, so it reports "missing ;", acts as if the semicolon had been present, and continues. No tokens are skipped, so no errors are lost.

The danger is that a repair can make things worse. A careless choice may loop forever (inserting a token that is never consumed) or produce a cascade of its own. Compilers that use phrase-level recovery restrict it to a few well-understood cases and fall back on panic mode for everything else.

Python Inserting a missing semicolon

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Compare the two runs. Without insertion, the parser reports "expected ;, found print" and points at the start of the next line. That is where the parser got stuck, but not where the programmer made the mistake. With insertion, the message says "missing ;" and the caret sits right after the last token of the unfinished statement, which is where the programmer needs to type. The program also continues parsing the second statement properly instead of discarding it.

Strategy 3: error productions and global correction

Two other strategies exist, and you should know them even though they are less common in hand-written parsers.

Error productions add rules to the grammar for typical mistakes. For example, a C-like compiler might include a production that accepts if x = 5 and reports "did you mean ==?" as a diagnostic. The parser then continues without any skipping. Parser generators support this with a special error token (see parser generators).

Global correction looks for the smallest number of insertions, deletions and replacements that turns the input into a valid program. It is a fine theoretical tool, and the algorithms are expensive, so production compilers do not use it. A minimal cost repair may also not be what the programmer meant.

Strategy Idea Strength Weakness
Panic mode Skip to a synchronizing token Simple, never loops if it always advances Can skip real errors
Phrase-level Insert, delete or replace a token locally Precise messages, nothing skipped Hard to make safe in general
Error productions Grammar rules for known mistakes Great diagnostics Larger grammar
Global correction Minimal edit distance to a valid program Elegant theory Too slow, may guess wrong

Writing diagnostics people can use

How you phrase a message matters as much as how you recover. Compare:

TEXT
Syntax error at token 14.
TEXT
line 3, column 9: expected ')', found ';'
   print a * (2 + 3;
                   ^

Good diagnostics usually share a few qualities:

  • Location first. File, line and column, with the offending line and a caret underneath. Keep the original text exactly as written.
  • Say what was expected and what was found, in terms of the user's language (";"), not the parser's internals ("token type 17").
  • Point at the cause. For a missing closing delimiter, mention where the opening one was.
  • Offer a hint when you are fairly sure. A common trick is to compare an unknown word against the keyword list using edit distance.
  • Report each problem once and stop after a sensible number of errors (many compilers stop at about 20).

The standard library has the pieces for a "did you mean" suggestion, as the next example shows.

Python A helpful 'did you mean' message

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A hint is offered only when there is a close match; otherwise listing the valid starters is more honest than a random guess. Never let a hint pretend to be certain.

Test your error messages like any other output. Keep a folder of small broken programs and the exact message each must produce, so a refactoring of the parser cannot silently make diagnostics worse. You will build such a harness in testing and debugging a compiler.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does panic-mode recovery do after a syntax error?

    Panic mode throws away input up to a token such as a semicolon or a statement keyword, which marks a place where parsing can safely restart.
  2. 2. Which is a typical synchronizing token for a statement-based language?

    Semicolons, closing braces and statement keywords mark statement boundaries, so resuming there lets the parser re-enter its normal loop.
  3. 3. Why is a message like 'missing ; (inserted)' with the caret after the previous token better than 'expected ; found print' pointing at the next line?

    The parser detects the problem at the next token, but the mistake is the missing symbol at the end of the previous statement. Phrase-level recovery can point at that spot.
  4. 4. What is a cascading error?

    Poor recovery leaves the parser in a confused state, so later, perfectly valid code triggers extra messages. Good recovery tries to report one message per real mistake.
  5. 5. Why do production compilers avoid global (minimum edit) error correction?

    Finding the minimal set of edits is costly for big inputs, and the cheapest fix can be different from the programmer's intention, giving confusing guidance.

Summary

  • Real programs often contain mistakes, so parsers should report several genuine errors per run and avoid cascades.
  • LL and LR parsers detect an error at the first token that cannot continue any valid program, which may be after the real mistake.
  • Panic mode skips to a synchronizing token (such as ; or a statement keyword); it is simple and robust as long as it always makes progress.
  • Phrase-level recovery repairs the input locally by inserting, deleting or replacing a token, giving precise messages for common slips.
  • Error productions and global correction exist, but hand-written parsers mostly combine panic mode with a few phrase-level repairs.
  • Good diagnostics give a location, the offending line with a caret, what was expected and found, and a hint only when it is likely right.

Next: Parser Generators and Tools