Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 40 ยท Data Pipelines and Saving Models

tf.data Pipelines

Build fast input pipelines with tf.data: from_tensor_slices, map, shuffle, batch, cache and prefetch. Includes plain-Python stand-ins that show how each step behaves.

A fast GPU is wasted if it waits for data. Calling model.fit(x, y) on NumPy arrays works for small data, but real projects have datasets that do not fit in memory, need decoding and augmentation, and must be shuffled and batched every epoch. tf.data is TensorFlow's input pipeline API for exactly that. A Dataset is a lazy description of a stream of elements plus a chain of transformations; the data is produced only when the training loop asks for it.

TensorFlow does not run in the browser here, so tf.data code is read-only. Every runnable block is a plain-Python or NumPy stand-in that behaves like the corresponding step, so you can see what shuffle, batch and map do to your data.

Creating a dataset

Python
import tensorflow as tf

features = tf.random.normal((1000, 8))
labels = tf.random.uniform((1000,), maxval=2, dtype=tf.int32)

ds = tf.data.Dataset.from_tensor_slices((features, labels))
for x, y in ds.take(2):
    print(x.shape, y.numpy())      # (8,) 1

from_tensor_slices slices along the first axis, so each element is one example. Other sources: from_generator, TextLineDataset, TFRecordDataset, and Keras helpers such as image_dataset_from_directory (see Loading Images and Text).

The standard pipeline

Python
ds = (
    tf.data.Dataset.from_tensor_slices((features, labels))
    .shuffle(buffer_size=1000)
    .map(preprocess, num_parallel_calls=tf.data.AUTOTUNE)
    .batch(32)
    .prefetch(tf.data.AUTOTUNE)
)
model.fit(ds, epochs=10)
flowchart LR
    A["Source<br/>files / arrays"] --> B["shuffle"]
    B --> C["map<br/>decode, augment"]
    C --> D["batch"]
    D --> E["prefetch"]
    E --> F["model.fit"]

The order matters: shuffle examples before batching, and prefetch last so the next batch is prepared while the model trains on the current one.

Here is what each step does, with stand-ins.

Batching

batch(n) groups n consecutive elements into one tensor with a new leading axis. The last batch may be smaller unless you pass drop_remainder=True.

Python batch with and without drop_remainder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Shuffling with a buffer

shuffle(buffer_size) does not load everything. It fills a buffer with the next buffer_size elements and each time picks one at random from it, then refills the slot. A buffer equal to the dataset size is a perfect shuffle; a small buffer only shuffles locally.

Python Buffer shuffle

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With buffer size 1 nothing moves. Sorted data (say, all of class 0 and then all of class 1 on disk) needs a buffer at least as large as the sorted runs, or batches will be dominated by one class. Also reshuffle_each_iteration=True (the default) gives a new order every epoch.

map: per-example transformations

map(fn) applies a function to every element, usually decoding, resizing, normalizing or augmenting. With num_parallel_calls=tf.data.AUTOTUNE several elements are processed at once on CPU threads.

Python map as preprocessing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Functions passed to map are traced into a TensorFlow graph, so use TensorFlow operations (tf.cast, tf.image.resize) inside them rather than arbitrary Python such as print or NumPy calls. Python side effects run only once at trace time.

cache and prefetch

  • cache() stores the elements after the first epoch (in memory, or on disk with cache("file")) so expensive decoding happens once. Put it before random augmentation, otherwise the same augmented image repeats.
  • prefetch(AUTOTUNE) overlaps preparing batch n+1 with training on batch n. It is almost always free speed.

This timing stand-in shows the effect: loading takes 3 units and training 3 units per batch. Done serially the cost adds up; overlapped, the slower of the two sets the pace.

Python Why prefetch helps

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other useful methods

Method Purpose
take(n) / skip(n) First n elements, or all but the first n (train/validation splits)
repeat(k) Repeat the dataset k times (or forever)
filter(fn) Keep elements where fn returns True
zip, concatenate Combine datasets
padded_batch Batch variable-length sequences, padding each batch to its longest element
window Sliding windows over a series
snapshot / cache Save processed data
Python Sliding windows for time series

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Keras also offers keras.utils.timeseries_dataset_from_array for this exact task, and fit() accepts a tf.data.Dataset directly, so you do not pass batch_size or y separately.

Common mistakes

  • Batching before shuffling, which shuffles whole batches rather than examples.
  • A shuffle buffer far smaller than the data when it is stored in sorted order.
  • Calling .cache() after random augmentation.
  • Forgetting .prefetch().
  • Splitting train and validation after shuffling with reshuffle_each_iteration=True, which lets examples leak between sets. Split first, then shuffle only the training dataset.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which order is generally correct?

    Shuffle individual examples, transform them, batch, and prefetch last so the pipeline works ahead of training.
  2. 2. What does shuffle(buffer_size=1) do?

    With a buffer of one element there is nothing to choose from, so order is preserved.
  3. 3. Why put cache() before random augmentation?

    Cached elements are reused exactly, so anything random after the cache stays random, but anything random before it would be frozen.
  4. 4. What does prefetch(tf.data.AUTOTUNE) do?

    It overlaps input preparation with training so the accelerator is not idle.

Summary

  • A tf.data.Dataset is a lazy stream of elements plus transformations.
  • Typical chain: from_tensor_slices, shuffle, map, batch, prefetch.
  • shuffle uses a buffer; make it large enough for how the data is ordered.
  • map(..., num_parallel_calls=AUTOTUNE) preprocesses in parallel using TensorFlow ops.
  • cache before augmentation and prefetch last remove most input bottlenecks.

Next: Loading Images and Text