Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 40 ยท Getting Started

Functional Programming in JAX

Learn why JAX needs pure functions, what side effects break, how to pass state explicitly, and how this style makes grad, jit and vmap safe to combine.

JAX works best when your code is written in a functional style: functions take inputs and return outputs, and nothing else changes. This is not a stylistic preference. grad, jit and vmap all work by running your function in a special mode and recording what it does. If the function secretly depends on, or modifies, state outside itself, the recording and the real behavior can disagree.

In this lesson you will learn what "pure" means, why impure code breaks under transformations, and the main technique for handling state: pass it in, return it back out. The runnable editors are NumPy-only stand-ins that illustrate the ideas (see What Is JAX? for why).

What is a pure function?

A function is pure when:

  1. Its return value depends only on its arguments (no hidden inputs: globals, the clock, files, random state).
  2. It has no side effects: it does not modify its arguments, globals, files or the screen.

Pure: def area(r): return 3.14159 * r * r. Impure: a function that increments a global counter, appends to a list, calls print, or uses np.random.rand().

Python Stand-in: pure and impure functions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

impure(3) returns a different answer on each call. A transformation like jit assumes it can run your function once, record the operations, and reuse that recording. For impure the recording would freeze whatever counter happened to be, and later calls would give stale answers.

What goes wrong in real JAX

Here are the classic failures, in read-only form:

Python
import jax
import jax.numpy as jnp

counter = 0

@jax.jit
def f(x):
    global counter
    counter += 1          # runs ONLY while JAX traces the function
    print("tracing!")     # also runs only during tracing
    return x + counter

print(f(1.0))   # prints "tracing!" then 2.0
print(f(1.0))   # no "tracing!": reuses the compiled code, still 2.0
print(f(2.0))   # a float again -> same compiled code: 3.0

The Python side effects (changing counter, calling print) happen during tracing, the first call, and are then skipped. The value of counter was baked into the compiled code as a constant. You will study tracing properly in Tracing and Static Arguments. For now, the rule is: if a function must behave the same every time, make it pure.

jax.debug.print exists for printing from inside jitted code; it is a deliberate exception that JAX handles for you.

Mutation of arguments

Because JAX arrays are immutable, in-place updates are rejected, which enforces purity for array arguments. You return a new array instead:

Python
def normalize(v):
    return v / jnp.linalg.norm(v)    # returns a new array

For Python containers such as lists and dicts, JAX cannot stop you from mutating them, so you must be careful yourself.

Handling state: pass it in, return it out

Real programs have state: model weights, optimizer momentum, random keys, running statistics. In JAX that state lives outside your functions and is threaded through them explicitly.

Compare an object-oriented counter with a functional one:

Python Stand-in: explicit state threading

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The functional versions look more verbose, but they have a big advantage: tick and sgd_step can be jitted, differentiated and vectorized, since everything they depend on and produce is visible in their signature.

This pattern is why JAX random numbers use explicit keys (new_key, subkey = split(key)), why Flax models separate parameters from the model definition, and why optimizers in Optax carry an explicit opt_state.

flowchart LR
    S0["state_t"] --> F["pure function"]
    I["inputs"] --> F
    F --> S1["state_t+1"]
    F --> O["outputs"]

A pure step takes the old state and inputs and returns the new state and outputs. Nothing else changes.

Python control flow and containers

Pure does not mean "no loops". You can use for loops, if statements on Python values, and recursion. The constraint is about what changes outside the function, and (under jit) about whether control flow depends on array values; we handle that in jit basics and lax control flow.

Why the restriction pays off

  • grad can differentiate a pure function by following its operations exactly.
  • jit can reuse a compiled version safely.
  • vmap can run many copies "at once" because copies cannot interfere with each other.
  • Parallelism is easier: without shared mutable state there are no data races.
  • Testing is easier: same inputs, same outputs.

Practical checklist

Before wrapping a function in a transformation, ask:

  1. Does it read any global that can change? Pass it as an argument.
  2. Does it write to a global, file or list? Return the data instead.
  3. Does it use random numbers? Take a key argument.
  4. Does it print for debugging? Use jax.debug.print or print outside the function.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. Which function is pure?

    Only the second depends solely on its argument and changes nothing outside. The others use or modify global state, randomness, or output.
  2. 2. A jitted function prints text and increments a global. What happens on the second call with the same input shape?

    Python side effects run during tracing; later calls reuse the compiled code, which contains only the array operations.
  3. 3. How is state handled in a functional style?

    Threading state through inputs and outputs keeps functions pure while still letting programs evolve over time.

Summary

  • Pure functions depend only on arguments and have no side effects.
  • Transformations record your function; hidden state or side effects lead to stale or surprising results.
  • Pass state in and return new state out (parameters, optimizer state, random keys).
  • jax.debug.print is the sanctioned way to print inside jitted code.
  • This discipline is what makes grad, jit and vmap composable.

Next: JAX Arrays and Data Types