Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 40 ยท Autodiff with grad

stop_gradient and Custom Derivatives

Control autodiff in JAX: block gradients with stop_gradient, fix numerically unstable derivatives with custom_jvp, and define your own backward pass with custom_vjp.

Automatic differentiation is mechanical: it applies the chain rule to whatever operations your function performs. Most of the time that is exactly what you want. Sometimes it is not. You may want a value to act as a constant during differentiation, or the "obvious" derivative may be numerically unstable, or you may want a different gradient than the true one (a trick used in quantized networks).

JAX gives you three tools: lax.stop_gradient, jax.custom_jvp and jax.custom_vjp. The JAX code here is read-only (see What Is JAX?); runnable stand-ins show the effects with plain Python.

stop_gradient: treat a value as a constant

jax.lax.stop_gradient(x) returns x unchanged in the forward pass, but autodiff treats it as a constant: its derivative is zero.

Python
import jax
import jax.numpy as jnp
from jax import lax

f = lambda x: x * lax.stop_gradient(x)     # forward value: x * x

f(3.0)               # 9.0
jax.grad(f)(3.0)     # 3.0   not 6.0, only the first factor is differentiated

Where is this useful?

  • Target networks and bootstrapping. In reinforcement learning the target r + gamma * V(s') is computed by the same network you are training. Wrapping it in stop_gradient stops the loss from "chasing its own tail".
  • Normalizing statistics. When you divide by a batch norm and want the norm to count as a constant.
  • Straight-through estimators (below).
  • Freezing part of a model without removing it from the function.
Python Stand-in: a dual number with stop_gradient

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Straight-through estimators

Rounding has a derivative of zero almost everywhere, so a network that rounds activations cannot learn through it. The straight-through trick rounds in the forward pass but pretends the operation was the identity in the backward pass:

Python
def round_ste(x):
    return x + lax.stop_gradient(jnp.round(x) - x)

round_ste(2.7)               # 3.0
jax.grad(round_ste)(2.7)     # 1.0, as if it were the identity

Read it algebraically: x + (round(x) - x) is round(x), so the value is the rounded number; but the bracket is a constant to autodiff, so the derivative is that of x, which is 1.

When the true derivative is bad: custom_jvp

Some functions have a correct derivative that is numerically unstable when computed step by step. The classic example is log(1 + exp(x)) (softplus) or log1pexp: for large x, exp(x) overflows, and a naive derivative produces inf / inf = nan.

You fix this by telling JAX the derivative yourself with jax.custom_jvp. A JVP (Jacobian-vector product) rule maps a point and a tangent to the output and its tangent:

Python
@jax.custom_jvp
def log1pexp(x):
    return jnp.log1p(jnp.exp(x))

@log1pexp.defjvp
def log1pexp_jvp(primals, tangents):
    (x,), (t,) = primals, tangents
    ans = log1pexp(x)
    # derivative is sigmoid(x) = 1 / (1 + exp(-x)): stable for large |x|
    return ans, t * (1.0 / (1.0 + jnp.exp(-x)))

JAX derives the reverse-mode gradient from the JVP rule automatically (by transposing it), so one rule supports grad, jacfwd, jacrev and higher derivatives. The rule is also used under vmap and jit.

Python Stand-in: naive derivative overflows, custom rule does not

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Changing the backward pass: custom_vjp

custom_jvp is the right tool when you can write the derivative as a forward-mode rule. jax.custom_vjp lets you write the reverse-mode rule directly, in two parts: a forward function that returns the output plus residuals (values saved for later), and a backward function that maps the output cotangent to input cotangents.

Python
@jax.custom_vjp
def clip_gradient(x):          # identity forward, but clips the gradient flowing back
    return x

def clip_fwd(x):
    return x, None             # no residuals needed

def clip_bwd(_, g):
    return (jnp.clip(g, -1.0, 1.0),)    # one cotangent per input

clip_gradient.defvjp(clip_fwd, clip_bwd)

jax.grad(lambda x: 10.0 * clip_gradient(x))(2.0)     # 1.0, not 10.0

The forward value is exactly x; only the gradient is altered. Typical uses are gradient clipping layers, gradient reversal in adversarial training, and memory-saving rules where recomputing a value in the backward pass is cheaper than storing it.

flowchart LR
    X["x"] --> F["custom forward"] --> Y["output"]
    F -. "residuals" .-> B["custom backward"]
    G["incoming gradient"] --> B --> DX["gradient for x"]

A custom VJP splits a function into a forward step and a backward step linked by residuals.

Python Stand-in: forward and backward by hand

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Which tool when?

Need Tool
Treat a value as a constant lax.stop_gradient
Different gradient, same value (straight-through) x + stop_gradient(f(x) - x)
Stable or cheaper forward-mode rule jax.custom_jvp
Explicit backward pass, residuals, gradient surgery jax.custom_vjp

A custom rule replaces JAX's derivative with yours, and JAX will not check it. Compare against finite differences (grad Basics) or jax.test_util.check_grads whenever you write one.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does lax.stop_gradient(x) do?

    The forward value is unchanged; the derivative contribution is zero.
  2. 2. What does the straight-through estimator x + stop_gradient(round(x) - x) compute?

    Algebraically it equals round(x), but the bracket is constant to autodiff, leaving d/dx x = 1.
  3. 3. When is custom_jvp most useful?

    The softplus example shows a correct but unstable naive derivative; a custom rule uses a stable formula.
  4. 4. What are the residuals in custom_vjp?

    The forward function returns them with the output; the backward function receives them with the cotangent.
  5. 5. How should you check a custom derivative?

    JAX does not verify custom rules, so gradient checking is the standard test.

Summary

  • lax.stop_gradient keeps the value but zeroes the derivative contribution.
  • The straight-through trick x + stop_gradient(f(x) - x) gives non-differentiable operations a usable gradient.
  • custom_jvp supplies a forward-mode rule; JAX derives reverse mode from it.
  • custom_vjp supplies a forward step with residuals and an explicit backward step.
  • JAX trusts custom rules, so test them numerically.

Next: Forward and Reverse Mode: jvp and vjp