Skip to content
Tutorials Center
My learning

Topics

Lesson 16 of 40 ยท jit and XLA

Tracing and Static Arguments

Understand how jax.jit traces functions with abstract values, why shapes must be known, what a tracer is, and when to use static_argnums or static_argnames.

When you call a jitted function for the first time, JAX does not run your code on real numbers. It runs it on tracers: placeholder objects that know the shape and dtype of an argument but not its value. Every operation applied to a tracer is recorded, and the recording becomes the compiled program. Understanding this one mechanism explains how jit works, what it cannot do and how static_argnums fixes the common problems.

JAX cannot run in the in-browser runtime (see What Is JAX?); JAX code is read-only and the editors use plain-Python stand-ins that model tracing.

What a tracer is

Python
import jax
import jax.numpy as jnp

@jax.jit
def f(x):
    print("x is:", x)          # runs during tracing only
    return x * 2

f(jnp.ones(3))
# x is: Traced<ShapedArray(float32[3])>with<DynamicJaxprTrace>

The printed object is not an array of ones. It says "a float32 array of shape (3,)" and nothing more. JAX passed it through x * 2, noted a multiplication, and moved on. On the second call with the same shape and dtype the print does not happen at all, because the compiled program is reused (jit Basics).

You can ask a tracer for its shape and dtype, because they are known:

Python
@jax.jit
def g(x):
    n = x.shape[0]            # a plain Python int: fine
    return x.reshape(n // 2, 2)

You cannot ask for its value.

Python Stand-in: a tracer that knows shape but not value

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The error you will meet most often

Branching on the value of an argument fails under jit:

Python
@jax.jit
def clip_positive(x):
    if x > 0:                 # TracerBoolConversionError
        return x
    return 0.0

JAX has to compile one program for every value of x; a Python if would choose a branch at trace time, from a value that does not exist yet. The same applies to while x < 10, to using a tracer as a list index, as range(n), or as a shape. The fixes, in order of preference:

  1. Use a vectorized formula: jnp.where(x > 0, x, 0.0).
  2. Use the structured control flow of lax.cond, lax.scan and lax.while_loop (jit with Control Flow).
  3. Mark the argument as static, so it is an ordinary Python value at trace time (below).

Static arguments

Some arguments are not data at all: a layer count, a mode string, an axis number, a boolean flag. These are naturally Python values. Marking them static keeps them out of the tracing machinery:

Python
from functools import partial

@partial(jax.jit, static_argnames=("n",))
def power(x, n):
    result = x
    for _ in range(n - 1):        # n is a Python int, so range works
        result = result * x
    return result

power(jnp.array([1.0, 2.0, 3.0]), 3)       # [1., 8., 27.]
power(jnp.array([1.0, 2.0, 3.0]), 4)       # compiles a NEW program for n = 4

Rules for static arguments:

  • A static argument becomes part of the cache key, so every distinct value compiles a separate program. Use them for a small set of values (modes, depths), not for a number that changes constantly.
  • Static values must be hashable (ints, strings, tuples, frozen dataclasses; not lists or arrays).
  • static_argnums=(1,) and static_argnames=("n",) do the same thing; names are clearer.
Python Stand-in: static values are part of the cache key

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Shapes must be known at compile time

XLA needs fixed shapes. Anything whose output shape depends on values cannot be jitted:

Python
@jax.jit
def positives(x):
    return x[x > 0]           # shape depends on the data: error under jit

Workarounds are to keep the full shape and mask (jnp.where(x > 0, x, 0.0)), to sum with a weight mask, or to pass a static size= to functions that support it, for example jnp.nonzero(mask, size=10), which pads or truncates to a fixed length. Outside jit the same expression works, because values are known.

What is allowed on a tracer

Operation Under jit
x.shape, x.dtype, x.ndim fine (static information)
arithmetic, jnp functions fine
x[2:5], x[i] with Python ints fine
x[mask] with a boolean array error (data-dependent shape)
if x > 0, while x < n error; use where or lax
range(x), jnp.zeros(x) with traced x error; make x static
float(x), int(x), x.item() error (needs a value)

Seeing the difference between shape and value

A subtle consequence: shape-dependent Python code is fine, because shapes are static, so loops over axes or layers unrolled at trace time are normal and compile into straight-line code:

Python
@jax.jit
def sum_rows(x):
    total = jnp.zeros(x.shape[1])
    for i in range(x.shape[0]):       # unrolled: x.shape[0] is a Python int
        total = total + x[i]
    return total

For long loops this unrolls into a huge program and slow compilation; replace it with lax.scan or jnp.sum(x, axis=0).

A good mental model: shapes and static arguments are decided when you trace; values are decided when you run. Whatever you want the Python code to branch on must be in the first group.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does a tracer carry?

    Tracing runs the function on abstract values so JAX can record the operations.
  2. 2. Why does 'if x > 0:' fail on a traced argument?

    A TracerBoolConversionError says exactly this: the boolean's value is unknown while tracing.
  3. 3. What must be true of static arguments?

    They form part of the cache key, so they need to be hashable and take few distinct values.
  4. 4. Which expression has a data-dependent output shape and therefore fails under jit?

    Boolean masking makes the output length depend on the values.
  5. 5. Is 'for i in range(x.shape[0])' allowed inside a jitted function?

    x.shape[0] is a Python int at trace time. Very long loops should use lax.scan to avoid huge programs.

Summary

  • jit traces your function with tracers that know shape and dtype, not values.
  • Python control flow on values fails; use jnp.where, lax control flow or static arguments.
  • static_argnums / static_argnames make selected arguments ordinary Python values, part of the cache key.
  • Output shapes must not depend on data values.
  • Shapes and static arguments are trace-time information; everything else is run-time data.

Next: jit with Control Flow and Recompilation