Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 40 ยท jit and XLA

jit Pitfalls and Side Effects

Avoid the classic jax.jit bugs: prints that run once, mutated globals, captured constants, impure functions and in-place updates. Learn jax.debug.print and checkify.

jax.jit assumes your function is pure: its output depends only on its arguments, and it does nothing else. When that assumption holds, compilation is invisible. When it does not, you get behaviour that looks like a bug: a print that fires once, a counter that never moves, a global that keeps its old value. This lesson lists the typical traps and shows the correct patterns, building on Functional Programming in JAX and Tracing and Static Arguments.

JAX cannot run in the in-browser runtime (see What Is JAX?); JAX code is read-only and the editors use plain-Python stand-ins that reproduce each effect exactly.

Pitfall 1: print runs only during tracing

Python
@jax.jit
def f(x):
    print("called with", x)
    return x + 1

f(1.0)     # prints: called with Traced<ShapedArray(float32[], weak_type=True)>
f(2.0)     # prints nothing: the compiled program has no print in it

The Python print executes while JAX traces, and the compiled executable contains only array operations. To print runtime values from compiled code use jax.debug.print, which becomes part of the program:

Python
@jax.jit
def f(x):
    jax.debug.print("x = {x}", x=x)      # prints on every call, with the real value
    return x + 1
Python Stand-in: why a print vanishes on later calls

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pitfall 2: global state and mutation

Python
counter = 0

@jax.jit
def f(x):
    global counter
    counter += 1          # happens once, at trace time
    return x * 2

f(1.0); f(2.0); f(3.0)
print(counter)            # 1, not 3

Appending to a list, updating a dictionary, writing to a file, incrementing a counter: all run only during tracing. Worse, leaking a tracer into global state (cache.append(x)) stores an object that is invalid after the trace and raises UnexpectedTracerError later.

The fix is to carry state through arguments and return values:

Python
@jax.jit
def f(x, counter):
    return x * 2, counter + 1

y, counter = f(1.0, 0)
y, counter = f(2.0, counter)

This is the same idea as the explicit RNG keys in PRNG Keys and split.

Pitfall 3: globals are captured as constants

A global read inside a jitted function is baked into the program at trace time. Changing it afterwards has no effect, because the function is not retraced:

Python
scale = 2.0

@jax.jit
def f(x):
    return x * scale

f(1.0)       # 2.0
scale = 10.0
f(1.0)       # still 2.0: the old value is part of the compiled program

Pass scale as an argument, or use functools.partial to build a new jitted function when it changes. The same applies to large arrays captured by closure: they are embedded as constants in the compiled program, bloating it and slowing compilation. Pass big arrays as arguments.

Python Stand-in: a captured global is frozen at trace time

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pitfall 4: impure functions and hidden inputs

Functions that read the clock, a file, environment variables or NumPy's global RNG are impure: they return different results for the same arguments, and jit caches the result of the first trace.

Python
@jax.jit
def noisy(x):
    return x + np.random.randn()      # the random number is chosen ONCE at trace time

Use jax.random with an explicit key passed as an argument (Random Sampling Patterns). Every call then has fresh numbers because the key changes, and jit is still valid.

Pitfall 5: in-place mutation

JAX arrays are immutable (Immutability and .at Updates), so x[0] = 1.0 raises an error. Use x = x.at[0].set(1.0). Inside jit the compiler turns this into an in-place update when it is safe, so it is not as costly as it looks. A related trap: mutating an argument that is a Python list or dict does not affect the caller's object in the way you expect, because the function sees tracers. Return new containers instead.

Pitfall 6: Python values that look like arrays

Python
@jax.jit
def f(x):
    return x * int(x.shape[0])        # fine: shape is static

@jax.jit
def g(x):
    return x * float(x[0])            # error: converts a tracer to a Python float

float(x), int(x), x.item(), np.asarray(x) and bool(x) all need a concrete value. Keep the computation in jnp until you leave the jitted function.

Debugging jitted code

Tool Use
jax.debug.print print runtime values from inside compiled code
jax.debug.breakpoint() interactive breakpoint inside jit
with jax.disable_jit(): run everything as plain Python, so ordinary print and pdb work
jax.config.update("jax_debug_nans", True) stops at the operation that produced a nan
jax.experimental.checkify turns runtime checks (index bounds, nan, user asserts) into returned errors
jax.make_jaxpr(f)(x) inspect what was traced (Inspecting jaxprs and HLO)
Python
with jax.disable_jit():
    result = f(x)          # debug as if there were no jit; remove when done

Python assert also runs only at trace time, and on traced values it raises an error. Use checkify when you need a real run-time check inside compiled code.

jax.disable_jit() makes code much slower and changes nothing about correctness. Use it to find a bug, not in production.

A pre-flight checklist

Before jitting a function, ask:

  1. Does it depend only on its arguments (plus constants)?
  2. Does it return everything it changes?
  3. Are all value-dependent decisions made with where, cond or scan?
  4. Are shapes stable across calls (jit with Control Flow)?
  5. Are big arrays passed in, not captured?

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does print(x) inside a jitted function show only on the first call?

    Use jax.debug.print to print real runtime values on every call.
  2. 2. A global scale = 2.0 is changed to 10.0 after the first call to a jitted function that reads it. What happens?

    Free variables are captured as constants at trace time; pass the value as an argument.
  3. 3. How should a jitted function keep a counter?

    State must flow through arguments and return values for the function to stay pure.
  4. 4. Which line fails on a traced value?

    Converting a tracer to a Python float needs a concrete value that does not exist while tracing.
  5. 5. What does jax.disable_jit() do?

    It is a debugging aid for finding problems in code that normally runs compiled.

Summary

  • A jitted function should be pure: outputs depend only on arguments, and all changes are returned.
  • Python side effects (print, list append, global writes) run only while tracing.
  • Free variables are frozen at trace time; pass changing values and large arrays as arguments.
  • Use jax.debug.print, jax.disable_jit(), debug_nans and checkify to debug.
  • Keep randomness explicit with jax.random keys.

Next: Async Dispatch and Benchmarking