Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 48 ยท Protocols, Errors and Tooling

Metaprogramming and Macros

Understand Elixir metaprogramming: the AST, quote and unquote, writing a simple defmacro, macro hygiene, __using__, and why you should avoid macros most of the time.

An Elixir program is data before it is a running program. When the compiler reads your source, it turns every expression into a small, regular data structure called the abstract syntax tree, or AST. A macro is a function that runs at compile time: it receives pieces of the AST, returns a new AST, and the compiler uses that result instead of the original code. That is metaprogramming, code that writes code.

You use macros every day without noticing. if, def, defmodule, |> and defstruct are all macros, written in Elixir itself on top of a tiny core. Knowing how they work makes the language feel less magical. It also teaches the most important lesson of this lesson: macros are powerful, and you should rarely write your own.

Elixir is not installed on this site, so all Elixir code here is read-only and the sample output was written from knowledge of how Elixir behaves. Metadata inside AST tuples (the middle element) varies between versions, so it is shown as [...] where it does not matter. The TryIts use a tiny JavaScript AST stand-in to demonstrate the idea of code as data and of a macro that rewrites it.

Code is data: quote

The quote special form returns the AST of the code inside it, without running it:

ELIXIR
quote do: 1 + 2
# {:+, [...], [1, 2]}

quote do: sum(1, 2, 3)
# {:sum, [...], [1, 2, 3]}

quote do: x * 10
# {:*, [...], [{:x, [...], Elixir}, 10]}

Almost every node is a three-element tuple {name, metadata, arguments}:

  • the name is an atom (:+, :sum) or another AST node (for calls like a.b()),
  • the metadata is a keyword list with information such as the line number,
  • the arguments are a list of more nodes.

A variable has an atom for its name and a context (a module name or nil) in place of the arguments, as in {:x, [...], Elixir}. Five kinds of values are their own AST: atoms, numbers, strings, lists and two-element tuples. So quote do: 1 is just 1.

Because the AST is ordinary data, you can pattern match on it, build it, and turn it back into source with Macro.to_string/1:

ELIXIR
ast = quote do: 1 + 2 * 3
Macro.to_string(ast)    # "1 + 2 * 3"

Code.eval_quoted/1 runs an AST (use it for exploring, not in production code):

ELIXIR
{result, _bindings} = Code.eval_quoted(ast)
result    # 7

What a macro does

Read The compiler parses your source into an AST.
Expand A macro gets that AST and returns new AST.
Compile The result is compiled like normal code.

All of this happens before your program starts running.

1 / 2

unquote: splicing values into a quote

Inside a quote, unquote pauses quoting and inserts a value or an AST built elsewhere:

ELIXIR
number = 5

quote do: 1 + unquote(number)
# {:+, [...], [1, 5]}

left = quote do: x
quote do: unquote(left) * 2
# {:*, [...], [{:x, [...], Elixir}, 2]}

Think of quote as a template and unquote as the holes in it. Without unquote, number would be inserted as a variable reference named number, not as the value 5.

Your first macro

A macro is defined with defmacro. Its arguments arrive already quoted, as ASTs, and it must return an AST. A macro must be defined in a module, and the caller must require that module (or import it) so the macro exists at compile time:

ELIXIR
defmodule MyMacros do
  defmacro unless_nil(value, do: block) do
    quote do
      if unquote(value) != nil do
        unquote(block)
      end
    end
  end
end

defmodule Demo do
  require MyMacros

  def run(name) do
    MyMacros.unless_nil name do
      IO.puts("Hello, #{name}")
    end
  end
end

When Demo compiles, the call to unless_nil is replaced by the if expression. At runtime there is no macro call, just an if. The inputs value and block are not values: inside the macro, value is the AST of name, and block is the AST of the IO.puts call. We spliced them into the template with unquote.

Why a macro and not a function?

A function receives values, evaluated first. The do block above must run only if the value is not nil, and a function would have evaluated it already. Macros can delay, repeat, skip or inspect code, which functions cannot. Here is a classic case, a macro that prints an expression together with its result:

ELIXIR
defmodule Debug do
  defmacro show(expression) do
    text = Macro.to_string(expression)

    quote do
      result = unquote(expression)
      IO.puts("#{unquote(text)} = #{inspect(result)}")
      result
    end
  end
end

# require Debug; Debug.show(1 + 2 * 3)   prints: 1 + 2 * 3 = 7

A function could not print 1 + 2 * 3, because it only sees 7. The macro sees the source text of the expression.

JS Code as data and a tiny macro, JavaScript stand-in

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Hygiene

Macros are hygienic: variables created inside a quote do not leak into the caller's code, and the caller's variables do not collide with the macro's. In the show macro above, the result variable lives in the macro's own context, so it cannot overwrite a result the caller already has.

ELIXIR
defmodule Hygiene do
  defmacro set_x do
    quote do
      x = 99
    end
  end
end

# x = 1
# Hygiene.set_x()
# x            # still 1; the macro's x was a different variable

When you truly want a macro to create a variable the caller can see, use var!/1:

ELIXIR
defmacro set_x_visibly do
  quote do
    var!(x) = 99
  end
end

This is rarely a good idea, because it makes the code harder to follow. Hygiene also applies to imports and aliases: a macro's call to IO.puts still works even if the caller aliased something else as IO. Hygiene prevents most surprises, but not all, and macros are still harder to debug than functions.

Expanding macros to see what they do

To understand a macro, expand it:

ELIXIR
ast = quote do: "42" |> String.to_integer()
Macro.expand_once(ast, __ENV__) |> Macro.to_string()
# "String.to_integer(\"42\")"

The pipe operator is a macro, and it expands into a plain nested call. Elixir also offers Macro.expand/2 to expand repeatedly, and Macro.to_string/1 to print the result as source. When a macro misbehaves, print its output.

use and __using__

use Module is a macro that calls Module.__using__/1 and pastes in whatever it returns. This is how use GenServer, use ExUnit.Case and Phoenix's use MyAppWeb, :controller inject default functions and imports:

ELIXIR
defmodule Greeting do
  defmacro __using__(opts) do
    greeting = Keyword.get(opts, :greeting, "Hello")

    quote do
      def greet(name), do: "#{unquote(greeting)}, #{name}!"
    end
  end
end

defmodule Friendly do
  use Greeting, greeting: "Hey"
end

Friendly.greet("Ada")    # "Hey, Ada!"

Keep __using__ small. A common good practice is to have it only import, alias or @behaviour things, instead of generating lots of functions that readers cannot find in the source.

When not to write a macro

Macros have real costs: they run at compile time, they are hard to read and debug, editors and static tools understand them less, and they can hide errors. The Elixir community has a clear rule of thumb:

  1. Use a function whenever one will do. Most of the time one will.
  2. Use a protocol or a behaviour for polymorphism (previous lesson).
  3. Reach for a macro only when you must change the structure of code: a new control construct, a DSL that needs the source text, or removing boilerplate that a function cannot.

Libraries such as Ecto (from p in Post, where: ...), Phoenix (get "/users", UserController, :index) and ExUnit (assert a == b printing both sides) use macros well because they give you a pleasant surface and clear errors. If you write one, document it, keep it short, and have it call a plain function as soon as possible, so the real work stays in code that is easy to test.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does quote do: 1 + 2 return?

    quote returns the AST: a {name, metadata, arguments} tuple like {:+, [...], [1, 2]}. The code is not run.
  2. 2. Why can a macro do something a function cannot?

    A function sees evaluated values. A macro sees the code itself, so it can delay, repeat, skip or inspect it.
  3. 3. What does hygiene mean for macros?

    Hygiene keeps the macro's variables, imports and aliases separate from the caller's. var!/1 lets you break the rule on purpose.
  4. 4. What does the caller need to do before using a macro from another module?

    Macros must exist at compile time, so the module has to be required (or imported, which also requires it).

Summary

  • Elixir code is data: quote returns the AST, usually {name, metadata, arguments}.
  • unquote splices values and other ASTs into a quote.
  • defmacro receives ASTs and returns an AST; the caller must require or import the module.
  • Macros are hygienic; var!/1 is the deliberate escape hatch.
  • Macro.to_string/1 and Macro.expand_once/2 show what a macro produces.
  • use calls __using__/1; keep it small.
  • Prefer functions, then protocols and behaviours, and write a macro only when you must change the structure of code.

Next, document and describe your code with types: Typespecs, Dialyzer and Documentation.