Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 40 ยท jit and XLA

jit Basics: Compiling Functions with XLA

Learn what jax.jit does, how XLA fuses operations into fast kernels, how to apply it as a decorator, and when compilation pays off. With runnable stand-ins that show the idea.

Python is flexible but slow: each operation in x * 2 + 1 is dispatched separately, allocates a temporary array and returns to the interpreter. JAX can avoid this overhead with jax.jit, which compiles a function once into a single optimized program for the CPU, GPU or TPU. The compiler is called XLA (Accelerated Linear Algebra).

This lesson shows how to use jit, what the compiler does for you and when it is worth it. JAX cannot run in the in-browser runtime (see What Is JAX?), so the JAX code is read-only; the editors use plain Python and NumPy stand-ins that illustrate the effects and print outputs that are certain.

Your first jitted function

Python
import jax
import jax.numpy as jnp

def selu(x, alpha=1.67, scale=1.05):
    return scale * jnp.where(x > 0, x, alpha * jnp.exp(x) - alpha)

selu_fast = jax.jit(selu)

x = jnp.arange(1_000_000, dtype=jnp.float32)
selu(x)         # runs op by op
selu_fast(x)    # first call: trace and compile; later calls reuse the result

The decorator form is the one you will use most:

Python
@jax.jit
def selu(x, alpha=1.67, scale=1.05):
    return scale * jnp.where(x > 0, x, alpha * jnp.exp(x) - alpha)

jit returns a function with the same signature and the same results (up to tiny floating-point differences). Nothing about how you call it changes.

What happens on the first call

flowchart LR
    A["Python function"] --> B["Trace with abstract values"]
    B --> C["jaxpr: JAX's own program"]
    C --> D["XLA compiles for the device"]
    D --> E["Cached executable"]
    E --> F["Fast calls"]

The first call pays for tracing and compiling; later calls with the same shapes and dtypes just run the executable.

  1. Tracing. JAX calls your Python function once with stand-in objects (tracers) that carry only the shape and dtype of the arguments, and records every operation they go through. See Tracing and Static Arguments.
  2. Compilation. The recorded program is handed to XLA, which optimizes it for the device.
  3. Caching. The compiled executable is stored under a key made of the argument shapes, dtypes and static values. Calling again with the same key skips steps 1 and 2.

The Python body of your function runs only during tracing. That single fact explains most surprises with jit, for example why print appears once (jit Pitfalls).

Python Stand-in: a cache keyed by shape and dtype

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Why compiled code is faster

XLA does more than remove interpreter overhead.

Fusion. Without jit the expression scale * jnp.where(x > 0, x, alpha * jnp.exp(x) - alpha) executes a separate kernel per operation and writes every intermediate array to memory. XLA fuses the chain into one kernel that reads x once and writes the result once. On accelerators memory traffic, not arithmetic, is usually the bottleneck, so fusion gives large speed-ups.

Whole-program optimization. Constants are folded, dead code is removed, layouts are chosen to suit the hardware, and operations can be reordered when it is safe.

Static shapes. Because shapes are fixed at compile time, XLA can plan memory and loops completely in advance.

Python Stand-in: counting trips to memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The code above only mimics what XLA does automatically. The point is the same: doing the same arithmetic with fewer trips to memory is faster.

jit with grad, vmap and other transformations

jit composes with everything in JAX, in any order:

Python
loss_grad = jax.jit(jax.grad(loss))            # compile the gradient
batched   = jax.jit(jax.vmap(f))               # compile a vectorized function
step      = jax.jit(lambda p, x: p - 0.1 * jax.grad(loss)(p, x))

Compiling the whole training step, forward pass, backward pass and parameter update, in one jit is the usual pattern. XLA then optimizes across all three stages.

Using jit as a decorator on methods and with arguments

jit accepts options. The most common are static_argnums/static_argnames (covered in the next lesson) and donate_argnums for reusing input memory. Calling jax.jit(f) on an already-jitted function is harmless.

For methods of classes, jit the pure function that the method calls, or use functools.partial(jax.jit, static_argnums=0) so that self is treated as static. Details come with custom pytrees.

When is jit worth it?

Use it for:

  • numerical code with array operations, especially chains of element-wise operations;
  • training steps and simulation steps called thousands of times;
  • anything running on a GPU or TPU, where launching kernels one by one is slow.

Skip it, or be careful, for:

  • code that is called once (compile time may exceed run time);
  • tiny computations dominated by Python overhead of the call itself;
  • code whose shapes change on every call (it recompiles each time);
  • functions that depend on Python values or print debug output (jit Pitfalls and Side Effects).

Measure properly: the first call includes compile time, and results are computed asynchronously, so use block_until_ready(). See Async Dispatch and Benchmarking.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. When does the Python body of a jitted function run?

    The body runs while JAX traces it; later calls with the same cache key run the compiled executable.
  2. 2. What does XLA fusion mean?

    A fused kernel reads inputs once and writes the final result once.
  3. 3. What triggers a recompilation?

    The cache key is made of shapes, dtypes and static values, not the data.
  4. 4. Which is the best use of jit?

    Compilation cost is paid once and amortized over many calls.
  5. 5. Can jit be combined with grad and vmap?

    JAX transformations compose; jit(grad(f)) and grad(jit(f)) both work.

Summary

  • jax.jit compiles a function with XLA; use it as a decorator.
  • The first call traces and compiles; later calls with the same shapes and dtypes reuse the executable.
  • XLA fuses operations and optimizes the whole program, which cuts memory traffic.
  • jit composes with grad, vmap and the rest; jit the whole training step.
  • Compile code that runs many times, with stable shapes.

Next: Tracing and Static Arguments