Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 40 ยท vmap and Vectorization

vmap Axes, Nesting and Pairwise Computations

Control which axis vmap maps with in_axes and out_axes, nest vmaps to build pairwise distance matrices, and know when to use jnp.vectorize or lax.map instead.

The previous lesson used the default behavior: map over axis 0 and stack results along axis 0. Real data does not always cooperate. Images may be stored as (height, width, batch), time series as (time, batch), and sometimes you want every combination of two sets of items, not matching pairs.

This lesson covers in_axes and out_axes in full, shows how nesting vmap calls produces all-pairs computations such as distance matrices, and finishes with two relatives of vmap: jnp.vectorize and lax.map.

As before, plain code blocks are real JAX for your machine; runnable blocks use NumPy and a small vmap stand-in that loops. The numbers are what JAX would give, the speed is not.

Mapping a different axis

in_axes accepts any axis number of the argument, including negative ones. The mapped axis is removed before the function sees the argument.

Python
import jax, jax.numpy as jnp

def total(row):                 # row has shape (4,)
    return jnp.sum(row)

M = jnp.arange(12.).reshape(3, 4)

print(jax.vmap(total, in_axes=0)(M))   # sums of rows    -> shape (3,)
print(jax.vmap(total, in_axes=1)(M))   # sums of columns -> shape (4,)

With in_axes=1, the function receives the columns of M; each one has shape (3,), so the function must work with that length. The data never moves in memory first: JAX adapts the batched operations to the chosen axis.

out_axes does the same for the output. By default the new axis is first. Setting out_axes=1 places it second, which is handy when later code expects (features, batch).

Python in_axes and out_axes (NumPy stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first two lines show row sums and column sums. The two shapes show that out_axes decides where the batch axis ends up, independently of in_axes.

Different axes for different arguments

Each argument gets its own entry. You can mix an axis number for one argument with another axis (or None) for the next.

Python
def combine(a, b):              # a: (3,), b: (3,)
    return a * b

A = jnp.ones((5, 3))            # batch on axis 0
B = jnp.ones((3, 5))            # batch on axis 1
out = jax.vmap(combine, in_axes=(0, 1))(A, B)
print(out.shape)                # (5, 3)

All mapped axes must have the same length (5 here). The unmapped dimension of each argument is whatever the function expects.

Pytrees as arguments

When an argument is a dict, tuple or list of arrays, a single integer in_axes applies to every array inside it. You can also give a structure with the same shape as the argument to map some leaves and not others. This is how one passes a model's parameters (shared) next to a batch of data (mapped), as shown in Per-Example Gradients.

Python
params = {"w": jnp.ones(3), "b": 0.5}          # shared, not mapped
x_batch = jnp.ones((10, 3))

f = lambda p, x: jnp.dot(p["w"], x) + p["b"]
out = jax.vmap(f, in_axes=(None, 0))(params, x_batch)
print(out.shape)                               # (10,)

None for the whole params tree means no leaf is mapped. If you only wanted to map w you would write in_axes=({"w": 0, "b": None}, 0).

Nesting vmap: every pair

One vmap pairs item i of the first input with item i of the second. To pair every item with every other item, nest two of them: the inner one maps over the second set while the outer one maps over the first. This is how a distance matrix is built.

flowchart TB
    D["dist(a, b): two single points -> one number"] --> I["inner = vmap(dist, in_axes=(None, 0))<br/>one a against all b"]
    I --> O["outer = vmap(inner, in_axes=(0, None))<br/>all a, each against all b"]
    O --> R["result shape (len(A), len(B))"]

The outer map walks the rows, the inner map walks the columns. No loop, no manual broadcasting.

Python
def dist(a, b):                      # two points of shape (2,)
    return jnp.sqrt(jnp.sum((a - b) ** 2))

pairwise = jax.vmap(jax.vmap(dist, in_axes=(None, 0)), in_axes=(0, None))

A = jnp.array([[0., 0.], [3., 4.]])
B = jnp.array([[0., 0.], [0., 1.], [6., 8.]])
print(pairwise(A, B))
# [[ 0.     1.    10.   ]
#  [ 5.     4.243  5.   ]]   (approximate; JAX prints more digits)
Python Pairwise distances with nested vmaps (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The broadcasting version needs A[:, None, :] and B[None, :, :], which hide the idea. The nested vmap states it directly: "for each a, for each b, compute dist". Swapping in_axes on the two levels transposes the result, so there is no axis arithmetic to get wrong.

Attention scores and other all-pairs patterns

The same pattern builds kernel matrices, similarity tables, and the score matrix of attention (every query against every key). Write the function for one query and one key, then nest.

Python A similarity matrix from one-pair code (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

axis_size and functions with no mapped input

If no argument is mapped (all in_axes are None), JAX cannot tell how big the batch should be. Pass axis_size. This is occasionally useful to run a function many times with different random keys, which is covered in Random Sampling Patterns.

Python
f = jax.vmap(lambda: jnp.float32(1.0), axis_size=4)
print(f())                      # [1. 1. 1. 1.]

Relatives: jnp.vectorize and lax.map

jnp.vectorize is NumPy's np.vectorize done properly. You give a function plus a signature that describes the core shapes, and it broadcasts over any extra leading axes by using vmap underneath.

Python
@jax.numpy.vectorize
def clipped_sum(x):             # no signature: element-wise, scalar in, scalar out
    return jnp.minimum(x, 10.0)

# With a signature: (n),(n)->()  means two vectors in, one scalar out
dot = jnp.vectorize(lambda a, b: jnp.dot(a, b), signature="(n),(n)->()")
print(dot(jnp.ones((4, 3)), jnp.ones((3,))).shape)   # (4,)

jax.lax.map(f, xs) also applies f along axis 0, but it runs the examples one after another in a compiled loop. It is slower than vmap but uses memory for one example at a time, which matters when a fully batched version would not fit on the device. A middle path, lax.map(..., batch_size=k) in recent versions, processes k examples at a time.

Tool Parallel over batch Memory When to use
vmap yes whole batch at once default choice
lax.map no (sequential) one example batch too big for memory
jnp.vectorize yes whole batch NumPy-style signatures

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does jax.vmap(f, in_axes=1)(M) do for a 2D array M?

    in_axes=1 maps along axis 1, so f receives each column in turn (with that axis removed).
  2. 2. How do you compute all pairs of distances between sets A and B with vmap?

    The inner vmap pairs one item of A with every item of B; the outer vmap repeats that for every item of A.
  3. 3. When is lax.map preferable to vmap?

    lax.map runs examples sequentially, so peak memory matches a single example.
  4. 4. What is axis_size for?

    If every in_axes entry is None, vmap cannot infer the batch size, so you provide it.

Summary

  • in_axes picks the mapped axis per argument (or None), out_axes places the new axis in the output.
  • A single integer applies to every leaf of a pytree argument; a matching structure maps leaves individually.
  • Nest two vmap calls with (None, 0) and (0, None) for all-pairs results such as distance matrices.
  • axis_size sets the batch size when nothing is mapped.
  • jnp.vectorize adds NumPy-style signatures; lax.map trades speed for memory.

Next: Per-Example Gradients with vmap and grad