Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 40 ยท Random Numbers and Pytrees

Custom Pytrees: Registering Your Own Classes

Make your own classes work with jit, grad and vmap: NamedTuple, register_dataclass, register_pytree_node, static fields, and the flatten and unflatten contract.

Dicts of arrays are fine for small examples, but real programs want named, typed containers: a TrainState holding parameters, optimizer state and a step counter; a Batch with images and labels; a Config that carries both arrays and a few settings. JAX will not look inside a custom class unless you tell it how, so by default such an object is a single opaque leaf, and passing it to jit or grad fails.

This lesson shows the three ways to make a class a pytree, starting with the easiest (none at all), and explains the one idea behind all of them: children that JAX transforms, and auxiliary data that stays fixed.

Plain blocks are real JAX. The runnable blocks write a miniature flatten/unflatten registry in pure Python so you can see the contract working. Real registration functions are shown in plain blocks.

Option 1: NamedTuple, no registration needed

A collections.namedtuple or typing.NamedTuple is already a pytree, with its fields as children. For simple immutable bundles, this is the cheapest solution.

Python
from typing import NamedTuple
import jax, jax.numpy as jnp

class Batch(NamedTuple):
    images: jnp.ndarray
    labels: jnp.ndarray

batch = Batch(jnp.ones((8, 28, 28)), jnp.zeros(8, dtype=jnp.int32))

@jax.jit
def n_examples(b: Batch):
    return b.images.shape[0]          # static shape, known during tracing

print(n_examples(batch))              # 8
print(jax.tree.leaves(batch)[0].shape)   # (8, 28, 28)

Downsides: NamedTuples are tuples, so len(batch) and indexing work (sometimes surprising), and they cannot have default method behavior you may want, or static fields.

Option 2: register_dataclass

For classes with methods, defaults and static settings, use a dataclass and tell JAX which fields are data (children) and which are metadata (static):

Python
from dataclasses import dataclass
import jax

@jax.tree_util.register_dataclass
@dataclass
class TrainState:
    params: dict
    opt_state: tuple
    step: jnp.ndarray

When every field is data, the bare decorator is enough in recent versions. When some fields are static, split them explicitly:

Python
@jax.tree_util.register_dataclass
@dataclass(frozen=True)
class Layer:
    w: jnp.ndarray                       # data: traced, differentiated, mapped
    b: jnp.ndarray
    activation: str = "relu"             # static: changes recompile, never traced

Depending on your JAX version, you may need to write the field lists out, as in register_dataclass(Layer, data_fields=["w", "b"], meta_fields=["activation"]). The idea does not change: data fields become children; meta fields go into the tree's static definition.

Option 3: register_pytree_node, full control

Underneath everything is a pair of functions. flatten turns an object into (children, aux_data); unflatten receives aux_data and a sequence of children and rebuilds the object.

Python
class Polynomial:
    def __init__(self, coeffs, name):
        self.coeffs = coeffs               # array: child
        self.name = name                   # string: auxiliary data

    def __call__(self, x):
        return sum(c * x ** i for i, c in enumerate(self.coeffs))

def flatten(p):
    return (p.coeffs,), p.name             # (children tuple, aux data)

def unflatten(aux, children):
    return Polynomial(children[0], aux)

jax.tree_util.register_pytree_node(Polynomial, flatten, unflatten)

Now jax.grad works on a function of a Polynomial, jit accepts it, and tree.map can add noise to its coefficients, all without a line of special handling. You can also decorate the class with @jax.tree_util.register_pytree_node_class and define tree_flatten and tree_unflatten methods instead.

flowchart LR
    O["Polynomial(coeffs, 'cubic')"] -->|flatten| C["children: (coeffs,)"]
    O -->|flatten| A["aux: 'cubic' (static, hashable)"]
    C --> T["JAX transforms children"]
    T -->|unflatten| N["Polynomial(new coeffs, 'cubic')"]
    A --> N

Children travel through grad, jit and vmap. Auxiliary data rides along unchanged and is part of the compilation cache key.

The contract

Two rules make custom pytrees behave:

  1. Children are arrays (or pytrees). They are what JAX traces, differentiates and maps.
  2. Auxiliary data must be hashable and comparable. It becomes part of the structure that jit compares on each call. A changed aux value means a different structure, so a recompile. Put names, flags and small integers there, never arrays.

And one rule of thumb: unflatten can receive non-array placeholders during transformations (tracers, object() sentinels, None). Your constructor must not validate or compute from children in __init__ in ways that assume real arrays, or transformations will break. Keep __init__ and unflatten thin.

A runnable miniature registry

The next example implements the same contract in pure Python, with a tiny registry, then uses it to apply one update to a custom class through a generic tree_map.

Python Register, flatten and rebuild a class (pure Python)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

tree_map scaled the coefficients by 10 and left the name alone, because only the children go through f while auxiliary data is passed back untouched. JAX does exactly this with your registered classes.

Static fields in practice

Suppose a layer has a Python callable as its activation. A function is not an array, so it cannot be a child; it belongs in aux data. Functions are hashable, so this works, and a different activation function compiles a separate program, which is the right behavior.

Python Static data selects behavior; children carry numbers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two layers share an activation but have different numbers, which is precisely the split JAX needs: numbers to transform, behavior to keep fixed.

When NOT to build a custom pytree

  • For quick scripts, a plain dict of arrays is clearer.
  • For models, you often do not hand-roll these at all: Flax modules produce plain dict parameter trees (see Flax Modules), and Optax state is already made of NamedTuples. Custom pytrees shine for your own state containers, such as a physics SimState or an RL Transition.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What happens when you pass an unregistered custom class instance to jax.grad or jit?

    JAX only looks inside known container types. An unregistered object is a leaf, and a non-array leaf cannot be traced.
  2. 2. In a flatten function, what belongs in auxiliary data?

    Aux data is compared when jit checks the tree structure, so it must be hashable and is expected to rarely change.
  3. 3. Which is the cheapest way to get a custom immutable container that works as a pytree?

    NamedTuples are pytrees automatically, with their fields as children.
  4. 4. What does changing a static (meta) field of a registered dataclass cause under jit?

    Static fields are part of the tree definition that jit keys its cache on.

Summary

  • Unregistered classes are opaque leaves; to look inside, make them pytrees.
  • NamedTuple needs no registration; register_dataclass suits stateful records; register_pytree_node gives full control.
  • flatten returns (children, aux_data); unflatten(aux, children) rebuilds the object.
  • Children are traced and differentiated; aux data is hashable, static, and part of the compilation cache key.
  • Keep constructors thin: they may receive tracers or placeholders during transformations.

Next: Flax Modules: init and apply