Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 40 ยท Advanced Topics

Custom Training Loops

Write your own training loop with tf.GradientTape, override train_step, use tf.function and metrics, and see the loop's structure with a NumPy gradient descent stand-in.

model.fit() handles batching, the forward pass, the loss, gradients, the optimizer update and metric tracking. For most work that is exactly what you want. But some models need something fit does not do: two optimizers (GANs), a loss that depends on several models, gradient surgery, reinforcement-learning updates or research ideas. For these you take control of the loop. Keras gives you two levels: override train_step (keep fit's conveniences, change one step) or write the whole loop yourself.

TensorFlow is not available in the browser runner. The TensorFlow code is read-only. The runnable blocks show the same loop structure in NumPy, with gradients written out by hand.

The anatomy of one training step

Every step does the same five things:

  1. Run the model forward on a batch inside a gradient recording scope.
  2. Compute the loss.
  3. Get the gradients of the loss with respect to the trainable weights.
  4. Apply the gradients with the optimizer.
  5. Update metrics.
flowchart LR
    A["batch (x, y)"] --> B["forward pass<br/>(recorded)"]
    B --> C["loss"]
    C --> D["gradients<br/>tape.gradient"]
    D --> E["optimizer.apply_gradients"]
    E --> F["updated weights"]
    C --> G["metrics"]

The tape records operations during the forward pass so it can replay them backwards.

The loop in NumPy

For linear regression the gradients are simple enough to write by hand, which makes the structure visible without any framework.

Python A training loop with manual gradients

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

GradientTape replaces step 3: you never derive the gradient formula yourself, TensorFlow derives it automatically for any model.

The same loop with GradientTape

Python
import tensorflow as tf
import keras

loss_fn = keras.losses.SparseCategoricalCrossentropy()
optimizer = keras.optimizers.Adam(1e-3)
train_acc = keras.metrics.SparseCategoricalAccuracy()
train_loss = keras.metrics.Mean()

@tf.function
def train_step(x, y):
    with tf.GradientTape() as tape:
        logits = model(x, training=True)          # training=True matters!
        loss = loss_fn(y, logits)
        loss += sum(model.losses)                  # layer regularization terms
    grads = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))
    train_acc.update_state(y, logits)
    train_loss.update_state(loss)

for epoch in range(5):
    train_acc.reset_state(); train_loss.reset_state()
    for x_batch, y_batch in train_ds:
        train_step(x_batch, y_batch)
    print(epoch, float(train_loss.result()), float(train_acc.result()))

Points to remember:

  • Call the model with training=True so dropout and batch norm act correctly.
  • tape.gradient(loss, variables) returns a list matching variables; None entries mean a variable was not connected to the loss.
  • @tf.function compiles the step into a graph. It is often several times faster than eager Python. Python side effects (printing, appending to lists) run only while tracing, so use tf.print and metric objects.
  • Metrics are stateful: update_state per batch, result() at the end of the epoch, reset_state() before the next.

Gradient clipping and other surgery

Since you hold the gradients as a list you can modify them before applying them:

Python
grads = tape.gradient(loss, model.trainable_variables)
grads, global_norm = tf.clip_by_global_norm(grads, 1.0)
optimizer.apply_gradients(zip(grads, model.trainable_variables))
Python Gradient clipping by global norm

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Clipping keeps the direction of the update and only shrinks its length, which tames exploding gradients in recurrent networks.

Overriding train_step

Often you do not want to rewrite the epoch loop, only the step. Subclass keras.Model and override train_step: fit still gives you batching, callbacks, progress bars and validation.

Python
class CustomModel(keras.Model):
    def train_step(self, data):
        x, y = data
        with tf.GradientTape() as tape:
            y_pred = self(x, training=True)
            loss = self.compute_loss(y=y, y_pred=y_pred)
        grads = tape.gradient(loss, self.trainable_variables)
        self.optimizer.apply_gradients(zip(grads, self.trainable_variables))
        for metric in self.metrics:
            if metric.name == "loss":
                metric.update_state(loss)
            else:
                metric.update_state(y, y_pred)
        return {m.name: m.result() for m in self.metrics}

inputs = keras.Input(shape=(8,))
outputs = keras.layers.Dense(1)(inputs)
model = CustomModel(inputs, outputs)
model.compile(optimizer="adam", loss="mse", metrics=["mae"])
model.fit(x, y, epochs=3)          # runs YOUR train_step

Return a dict of metric values; fit prints them. Override test_step the same way (without the tape) to customize validation.

Note that GradientTape is TensorFlow-specific. In Keras 3 the multi-backend way to write train_step uses the backend's own autodiff (jax.grad, torch autograd), so check the Keras guide "Customizing what happens in fit" for your backend. With the TensorFlow backend, the code above is correct.

When to write the full loop

Situation Approach
Standard supervised learning fit()
Different loss or extra logging in each step Override train_step
Two models or optimizers (GAN, actor-critic) Custom loop, or override with several optimizers
Complicated schedules, curriculum, research prototypes Custom loop

A custom loop gives total control but you re-implement callbacks, validation and progress reporting. Start with fit; step down only when you need to.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does tf.GradientTape do?

    Operations inside the tape's scope are recorded and replayed backwards to get gradients.
  2. 2. Why call model(x, training=True) in a custom loop?

    Without it, dropout is disabled and batch norm uses inference statistics.
  3. 3. What is true about Python print() inside a @tf.function?

    Python code in a traced function runs during tracing, not on every call of the compiled graph.
  4. 4. When is overriding train_step better than writing a whole loop?

    train_step customizes one step while fit keeps handling epochs, callbacks and validation.

Summary

  • A training step is: forward pass under a tape, loss, gradients, optimizer update, metrics.
  • GradientTape derives gradients automatically; tf.function compiles the step for speed.
  • Use training=True, reset metrics each epoch, and modify gradients (clipping) before applying.
  • Override train_step/test_step to change one step but keep fit.
  • Write a full custom loop only for multi-model or research-style training.

Next: Mixed Precision and Distributed Training