Skip to content
Tutorials Center
My learning

Topics

Lesson 108 of 132 ยท Advanced Rust

Macros in Practice: DSLs

Build real macro_rules! macros - a hand-written matches!, a struct builder generator, a tiny HTML DSL with a tt muncher - plus hygiene traps and when not to use macros.

The lesson Declarative Macros taught the mechanics: patterns, fragment specifiers, repetitions and recursion. Mechanics are only half of the craft. This lesson is about what people actually build with macro_rules!: macros that remove boilerplate, macros that give a library its own mini-syntax (a domain-specific language, or DSL), and the habits that keep such macros debuggable.

We will build four things step by step: a re-creation of the standard matches!, a macro that generates a struct together with its builder, a small HTML-like DSL that uses a tt muncher, and a macro that illustrates the most common hygiene trap. We finish with an honest list of cases where you should write a function instead.

A warm-up: rebuilding matches!

matches!(value, pattern) returns true when the value fits the pattern. It is short enough to rebuild completely, and it shows the pat fragment and the optional if guard.

Rust Your own matches!

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

Two details are worth noticing. $pattern:pat matches the whole pattern, including top-level | alternatives (since the 2021 edition). And $(if $guard:expr)? makes the guard optional: when the optional part is absent, the $(if $guard)? in the output disappears too. The $(,)? at the end accepts a trailing comma, a courtesy every list-like macro should offer.

Generating a struct and its builder

Rust has no named or default arguments, so types with many options use the builder pattern (Typestate and Builders). The builder is pure boilerplate: one field, one setter and one default per option. A macro can write it all from a single declaration.

Rust One declaration, a struct plus its builder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The macro takes attributes (#[derive(Debug)]) through $(#[$meta:meta])* and forwards them, so the caller keeps control of derives. The setter name is the field name, which is the style used across the ecosystem. Every repetition $( ... )+ ranges over the same list of fields, and the compiler requires that all repetitions inside one expansion have the same length, so the struct, the builder and build stay in sync automatically.

Real projects often use the derive_builder or bon crates for this, which are procedural macros (Procedural and Derive Macros) and can do more, for example required fields and validation. A macro_rules! builder is still a good tool when the shape is simple and you do not want a dependency.

A small DSL: an HTML-like tree

A DSL macro lets callers write something that looks like the thing it describes. We will aim for this:

TEXT
html! {
    div [class = "card"] {
        h1 { "Title" }
        p { "Some " b { "bold" } " text" }
    }
}

No commas, no builder calls, just the shape of the document. The tricky part is that the content of braces is a free-form list of items, and a single macro rule cannot loop over items of different shapes. The standard technique is a tt muncher: a recursive macro that takes the input as a list of token trees ($($rest:tt)*), peels off the first item with one rule, handles it, and calls itself on the rest. We accumulate the finished items in a bracketed list, which is the "accumulator" argument.

Rust html! with a tt muncher

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

Follow one expansion by hand. html! receives div [...] { h1 {...} p {...} }, builds an Element and hands the body to html_children!(@acc [] h1 {...} p {...}). The third rule matches h1 { "Hello" }, appends html!(h1 { "Hello" }) to the accumulator and recurses with p {...}. The first rule fires when no tokens are left and turns the accumulator into a vec!.

Three details make this robust:

  • Rule order. The literal rule comes before the element rule, so a string is never mistaken for an identifier. Put the most specific rule first and the catch-all last.
  • The @acc marker. An internal rule is distinguished by a token (@acc) that no user input would start with. Without it, internal calls could collide with the public syntax.
  • Two macros reference each other. html! is used inside html_children! and vice versa. Macros are resolved when expanded, so the order of definition inside one file is not a problem here, but a macro must be defined above the place where it is used when it is used in the same module as code (textual scope). Both are defined above main.

A tt muncher recurses once per item. The compiler stops at the recursion limit (128 by default; raise it with #![recursion_limit = "256"] in the crate root). Large inputs can also be slow to compile, because each step re-parses the remaining tokens, which makes the total work quadratic. For long lists, prefer a repetition ($( ... )*) whenever the items all have the same shape.

Reading errors from a DSL

What happens when the caller writes something the macro does not understand? Try an attribute without quotes:

Rust A DSL typo gives a macro error (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The message is "error: no rules expected main", followed by a note "while trying to match meta-variable $value:literal". The location points into your own call, which is helpful, but the text talks about the macro's internals. This is the reason to keep patterns simple: the more elaborate the grammar, the more cryptic the failures. You can improve things by adding a final catch-all rule that calls compile_error!("expected: tag!(name [key = \"value\", ...])"), which replaces the generic message with one you wrote.

Hygiene and the double-evaluation trap

Rust macros are hygienic: a local variable created inside a macro cannot clash with a caller's variable of the same name (see Declarative Macros). The flip side surprises people. A macro cannot use a caller's local variable unless the caller passes it in:

Rust A macro cannot see the caller's locals (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The compiler answers error[E0425]: cannot find value total in this scope, and adds the help "an identifier with the same name exists, but is not accessible due to macro hygiene", pointing at the caller's let total. The fix is to pass the name as an ident or the value as an expr.

The second classic trap is evaluating an expression twice. In C, macros paste text; in Rust, an expr fragment is a parsed expression, so square!(1 + 2) is correctly 9. But if you write $x * $x, the expression itself is still evaluated twice, along with its side effects:

Rust Double evaluation and the let fix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

Binding the argument to a let inside the macro, in a block {{ ... }}, is the habit that prevents this. It is exactly what assert_eq! does so that failure messages can print the evaluated values without evaluating them again.

Counting and generating enums: a muncher with no recursion

Not every list needs a muncher. When all items have the same shape, one repetition is enough, and you can build several outputs from the same input:

Rust An enum with names and a list of all variants

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

stringify!($variant) turns the identifier into a string literal at compile time. For a derive-like job such as this, a procedural macro would also work, but a macro_rules! version of 15 lines is easier to read and compiles faster.

Guidelines for macros that last

  1. Try a function, a generic or a closure first. A macro is the right tool only for what functions cannot do: variable arguments, new syntax, generating items (structs, impls, enums), capturing the text of an expression (stringify!, assert!) or the call site (line!, file!).
  2. Document the syntax with examples in a doc comment, since there is no signature to read.
  3. Accept trailing commas with $(,)?.
  4. Evaluate arguments once with a let.
  5. Use full paths ($crate::Node, ::std::vec::Vec) in exported macros so they work wherever they are called.
  6. Offer a compile_error! fallback for the most likely mistakes.
  7. Keep expansions small. Put the real logic in ordinary functions that the macro calls; functions are easier to test, debug and read in stack traces.
  8. Look at the expansion while developing (cargo expand, or -Zunpretty=expanded on nightly); our online editor shows only the final output.

When not to use a macro

  • To save a few keystrokes: a small generic function is clearer and type-checked at the definition site, not at every use.
  • For configuration or data: use a struct, a const table or a parsed file.
  • When the grammar needs more than a handful of rules: a procedural macro or a plain parser (see Text Formats and Parsing) gives better errors.
  • When IDE support matters: completion and "go to definition" inside macro input are weaker than in plain Rust.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In html_children!, what is the job of the @acc [...] argument?

    A tt muncher passes its partial result along as an accumulator; the @acc token distinguishes internal rules from the public syntax.
  2. 2. Why does the literal rule come before the identifier rule in a tt muncher?

    macro_rules! tries rules top to bottom. Putting the specific pattern first avoids a more general one swallowing the input.
  3. 3. What is wrong with ($x:expr) => { $x * $x } when called as square!(next())?

    Rust expr fragments keep their precedence, but the expression is still pasted twice. Bind it with let value = $x; first.
  4. 4. Why does the macro show_total!() fail to see the caller's local variable total?

    Local variables are hygienic. To use the caller's variable, pass it in as an argument.
  5. 5. Which is the best reason to choose a macro over a function?

    Macros earn their complexity when functions cannot do the job. Otherwise a function or generic is clearer and gives better errors.

Summary

  • macro_rules! is best for removing boilerplate (builders, enum helpers, repeated impls) and for small, regular DSLs.
  • Use $(,)? for trailing commas, $(#[$meta:meta])* to forward attributes, and $(...)? for optional parts.
  • A tt muncher peels one item at a time and passes the rest along with an accumulator; put specific rules first and mind the recursion limit.
  • Evaluate each argument once with a let; pass in every variable the macro needs.
  • Macro errors describe the macro's internals; add compile_error! fallbacks for friendlier messages.
  • Prefer functions and generics, and move logic out of the macro into ordinary code.

Next: Building Data Structures with Unsafe