Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 40 ยท vmap and Vectorization

vmap Basics: Write One Example, Run Many

Learn what jax.vmap does, how it turns a function for one example into a batched one, and how it compares with Python loops and manual broadcasting.

Most numerical code starts life as a function for a single input: the distance between two points, the loss on one training example, the prediction for one image. Real work needs that function applied to thousands of inputs at once. The classic fix is to rewrite the function by hand so it handles a leading batch dimension, and that rewrite is where shape bugs breed.

jax.vmap ("vectorizing map") removes the rewrite. You write the function for one example, and vmap returns a new function that takes a whole batch. This chapter shows how it works, how to control which axes are mapped, how it combines with grad, and where it stops being magic.

JAX itself cannot run in the browser: it needs a native compiled library (jaxlib) that no browser build provides. Code in plain code blocks is real JAX, written for your own machine (pip install jax). The runnable examples on this page use NumPy and a tiny hand-written stand-in for vmap, and each one is labelled. The stand-in shows the meaning of vmap (what comes out), not its speed. See the track overview for the full list of lessons.

The idea in one picture

vmap does not run a Python loop at call time. It traces your function once, then rewrites every operation inside it so that it works on the batch axis directly. The result is a single batched program, exactly what you would have written by hand.

flowchart LR
    F["f(x): works on ONE example<br/>x has shape (3,)"] --> V["vmap(f)"]
    V --> B["batched f: works on MANY<br/>X has shape (N, 3)"]
    B --> O["output has a new leading axis<br/>shape (N, ...)"]

The mapped axis appears in the input, travels through the function unseen, and reappears in the output.

Your first vmap

Here is the real JAX version. The function norm_squared is written for a single vector. Calling it on a matrix would give the wrong answer (it would sum everything), but vmap fixes that without changing a line of the function.

Python
import jax
import jax.numpy as jnp

def norm_squared(v):          # v has shape (3,)
    return jnp.sum(v ** 2)

points = jnp.array([[1., 2., 2.],
                    [0., 3., 4.],
                    [1., 0., 0.]])

batched = jax.vmap(norm_squared)
print(batched(points))        # [ 9. 25.  1.]

The same meaning, runnable with a stand-in. Our vmap below loops over the mapped axis and stacks the results. It gives identical numbers to the real one; it is only slower, and it cannot do what JIT compilation can.

Python A tiny vmap stand-in (NumPy)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both lines print the same three numbers. The hand-batched version needs you to remember axis=1; with vmap you never mention the batch axis at all.

Why not just a Python loop?

A loop such as [norm_squared(p) for p in points] gives the same values. The difference is cost and composability:

Approach Dispatch cost Compiles to one kernel? Composes with grad, jit
Python loop over examples one call per example no clumsy
Hand-batched rewrite one call yes yes, but you wrote it
jax.vmap one call yes yes, automatically

In JAX the real gain is that a vmapped function is one program. Put jax.jit around it and XLA fuses the whole batch into a few big kernels, which is what GPUs and TPUs like.

Unmapped arguments: in_axes=None

Functions usually take more than one argument. Often only some are batched: a thousand data points but one weight vector. in_axes has one entry per argument. 0 means "map over axis 0", and None means "do not map, give every example the same value".

Python
def predict(w, b, x):                 # one example x of shape (3,)
    return jnp.dot(w, x) + b

w = jnp.array([0.5, -1.0, 2.0])
b = 0.1
X = jnp.ones((100, 3))

batched_predict = jax.vmap(predict, in_axes=(None, None, 0))
print(batched_predict(w, b, X).shape)   # (100,)

Forgetting None and leaving the default in_axes=0 would try to map over the first axis of w as well, and JAX would raise an error about inconsistent axis sizes. That error is a feature: it catches shape mistakes immediately.

Python in_axes=None shares an argument (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

vmap versus broadcasting

NumPy already vectorizes many things through broadcasting, so why bother? Broadcasting only helps for element-wise operations and a few reductions. As soon as the per-example function contains steps that are awkward to batch (a matrix-vector product with a different matrix per example, an outer product, a small linear solve, a cumsum along one axis), the manual rewrite needs einsum or careful axis arguments. With vmap you keep the readable single-example version.

Python
def outer_then_trace(a, b):           # a, b: shape (4,)
    return jnp.trace(jnp.outer(a, b)) # one number

A = jnp.ones((5, 4))
B = jnp.ones((5, 4))
print(jax.vmap(outer_then_trace)(A, B).shape)   # (5,)

jnp.trace and jnp.outer were never written with a batch axis in mind, and they do not need to be.

What vmap returns

The output always has the mapped axis at position out_axes (default 0). Functions may return several values, or nested containers such as dicts, tuples and lists. vmap handles all of them; the structure is kept and every array leaf gets the extra axis. (These containers are called pytrees, covered in Pytrees Basics.)

Python Multiple and structured outputs (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Scalars become shape (3,) arrays and the two-element head becomes (3, 2): the batch axis is added in front of every leaf.

Common beginner mistakes

  • Passing a Python list. vmap maps over array axes. A list of arrays is a pytree whose leaves are mapped separately, which is rarely what you intend. Stack into one array first.
  • Mixing batch sizes. Every mapped argument must have the same size along its mapped axis (3 points and 4 labels is an error).
  • Expecting data-dependent shapes. The function must return the same shape for every example. A function that returns "the elements greater than zero" has a different length per example and cannot be batched.
  • Using Python side effects. print and list append inside a vmapped function run once, at trace time, with abstract values, not once per example.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does jax.vmap(f) return?

    vmap returns a new function. Called with batched inputs it behaves as if f ran on each example and the results were stacked along a new axis.
  2. 2. You have one weight vector w and a batch X of inputs. Which in_axes maps only X in f(w, x)?

    None means the argument is not mapped and every example receives the same value. 0 means map along axis 0.
  3. 3. Why is vmap usually better than a Python loop in JAX?

    A loop dispatches one operation per example. vmap rewrites the traced function to work on the whole batch at once.
  4. 4. Which function cannot be vmapped over examples?

    All examples must produce outputs with the same shape so they can be stacked.

Summary

  • vmap turns a function written for one example into one that handles a batch, without you touching the batch axis.
  • in_axes chooses which arguments are mapped (an integer) and which are shared (None); out_axes says where the new axis lands.
  • A vmapped function is one program, so it combines with jit and grad.
  • Output shapes must not depend on data, and arguments must agree on the mapped size.
  • Runnable examples here use a NumPy stand-in; the plain code blocks are real JAX.

Next: vmap Axes, Nesting and Pairwise Computations