Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 45 ยท Computer Vision

Image Classification Pipeline

Build an image classification pipeline: torchvision datasets, transforms, normalization, a small CNN, evaluation and a confusion matrix, with live stand-ins.

Image classification is the "hello world" of computer vision: given a picture, predict one label out of a fixed set. The model matters, but so does everything around it: loading the images, converting them to tensors, scaling the pixel values, batching, training, and finally measuring where the model goes wrong. In this lesson you will assemble that whole pipeline with torchvision, the companion library of datasets, transforms and pretrained models.

PyTorch and torchvision cannot run on this site (and downloading a dataset needs a network), so the PyTorch code is read-only. For live practice we use stand-ins: a small hand-written transform pipeline in NumPy, and scikit-learn's built-in 8 by 8 handwritten digits dataset with a small neural network and a confusion matrix. The structure of the pipeline is identical; only the library calls differ. You should know convolutions and CNNs and the training loop (or at least the loop pattern from Datasets and DataLoaders) before reading on.

The pipeline in five steps

Dataset Images plus labels, loaded lazily.
Transforms Resize, to tensor, normalize, augment.
DataLoader Shuffled mini-batches.
Model and loop CNN, loss, optimizer, epochs.

Then evaluate on data the model never trained on.

1 / 2

Datasets and transforms with torchvision

torchvision.datasets ships classic benchmarks (MNIST, Fashion-MNIST, CIFAR-10 and more) as ready-made Dataset objects, and torchvision.transforms provides the preprocessing steps. Composing them looks like this:

Python
import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets, transforms

train_tf = transforms.Compose([
    transforms.RandomAffine(degrees=10, translate=(0.1, 0.1)),  # augmentation
    transforms.ToTensor(),                       # PIL image -> float tensor in [0, 1], shape (1, 28, 28)
    transforms.Normalize((0.1307,), (0.3081,)),  # (pixel - mean) / std, per channel
])
test_tf = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,)),
])

train_ds = datasets.MNIST("data", train=True,  download=True, transform=train_tf)
test_ds  = datasets.MNIST("data", train=False, download=True, transform=test_tf)

train_loader = DataLoader(train_ds, batch_size=128, shuffle=True)
test_loader  = DataLoader(test_ds,  batch_size=256)

A few things deserve attention.

ToTensor converts a PIL image (or NumPy array) with integer pixels 0 to 255 into a float32 tensor in the range 0 to 1 and moves the channel axis to the front. Newer torchvision versions also offer the v2 API (torchvision.transforms.v2, with ToImage and ToDtype) which works on tensors, bounding boxes and masks; the idea is the same.

Normalize(mean, std) subtracts the mean and divides by the standard deviation of each channel. The numbers 0.1307 and 0.3081 are the mean and standard deviation of the MNIST training pixels. Centered, similarly scaled inputs make optimization better behaved, because gradient descent works poorly when features live on very different scales. For RGB photos you give three means and three stds; when you use pretrained models (see Transfer Learning and Fine-Tuning) you must use the statistics the model was trained with.

Augmentation belongs to training only. Random shifts, rotations and flips create new variants of each image every epoch, which fights overfitting. The test set must stay untouched, or your measurements become noisy and optimistic. Be careful which augmentations are label-preserving: flipping a photo of a cat is fine, flipping the digit 2 or 5 horizontally changes what it looks like.

The transform runs inside Dataset.__getitem__, so with num_workers > 0 the DataLoader applies it in parallel worker processes while the GPU trains on the previous batch.

A stand-in for the transform pipeline

Compose is just a list of functions applied in order. The editor builds a miniature version in NumPy: convert integer pixels to floats, normalize with statistics computed from the training images only, and apply a random shift as augmentation. The result shows what the model would actually see.

Python A tiny transform pipeline in NumPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The model and the training loop

The CNN from the previous lesson needs only small changes for a new dataset: the number of input channels and classes, and the size of the flattened features. Here is a variant with batch normalization and dropout, followed by the loop (the full anatomy is in The Complete Training Loop):

Python
device = "cuda" if torch.cuda.is_available() else "cpu"

model = nn.Sequential(
    nn.Conv2d(1, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(), nn.MaxPool2d(2),   # 14x14
    nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(), nn.MaxPool2d(2),  # 7x7
    nn.Flatten(), nn.Dropout(0.3), nn.Linear(64 * 7 * 7, 10),
).to(device)

loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3)

for epoch in range(5):
    model.train()
    for xb, yb in train_loader:
        xb, yb = xb.to(device), yb.to(device)
        optimizer.zero_grad()
        loss = loss_fn(model(xb), yb)
        loss.backward()
        optimizer.step()

    model.eval()
    correct = 0
    with torch.no_grad():
        for xb, yb in test_loader:
            xb, yb = xb.to(device), yb.to(device)
            correct += (model(xb).argmax(dim=1) == yb).sum().item()
    print(f"epoch {epoch + 1}: test accuracy {correct / len(test_ds):.3f}")

Three details matter. model.train() and model.eval() switch dropout and batch norm between training and inference behavior. Evaluation runs under torch.no_grad() so no graph is built. And the prediction is argmax over the logits: you never need softmax just to pick the winning class, because softmax does not change which score is largest.

With this setup a network of this size typically reaches about 99 percent test accuracy on MNIST within a few epochs, but treat that as a ballpark, not a promise: it depends on the seed, augmentation and optimizer.

Beyond accuracy: the confusion matrix

A single accuracy number hides which classes get mixed up. A confusion matrix counts, for every true class (rows), how often each class was predicted (columns). The diagonal holds correct predictions; large off-diagonal entries reveal specific confusions, such as 4 versus 9, or 3 versus 5. Per-class precision (of everything predicted as class c, how much was right) and recall (of everything truly class c, how much was found) come straight from its column and row.

In PyTorch you collect predictions during evaluation and then compute the matrix with NumPy or scikit-learn:

Python
all_preds, all_labels = [], []
model.eval()
with torch.no_grad():
    for xb, yb in test_loader:
        all_preds.append(model(xb.to(device)).argmax(dim=1).cpu())
        all_labels.append(yb)
preds = torch.cat(all_preds).numpy()
labels = torch.cat(all_labels).numpy()

from sklearn.metrics import confusion_matrix, classification_report
print(confusion_matrix(labels, preds))
print(classification_report(labels, preds))

Note the .cpu() before collecting: NumPy-based tools need CPU tensors. The next editor runs this exact evaluation recipe live, with a small neural network from scikit-learn standing in for the CNN, on the real 8 by 8 digits.

Python Digits classifier with a confusion matrix (scikit-learn stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common pitfalls

  • Normalizing with test statistics (or per-batch statistics) leaks information and makes train and test inputs inconsistent. Compute once on the training set and reuse.
  • Augmenting the test set makes results random and hard to compare. Keep test_tf deterministic.
  • Forgetting model.eval(): with dropout and batch norm active, predictions change between runs and accuracy drops.
  • Wrong tensor layout: a NumPy or Matplotlib image is height, width, channels; the model wants channels first. ToTensor handles this for PIL images, but your own arrays need permute.
  • Class imbalance: when one class dominates, accuracy looks great even if rare classes are never found. The confusion matrix and per-class recall expose this.

MNIST is a very easy benchmark. Do not judge a technique by a tiny gain there: use a harder dataset (Fashion-MNIST, CIFAR-10) or a held-out validation split before drawing conclusions.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does transforms.ToTensor() do to a PIL image with 0 to 255 integer pixels?

    ToTensor scales pixels to 0..1 and reorders the axes to (C, H, W). Normalization is a separate step, and device transfer is done with .to(device).
  2. 2. Where should random augmentation such as RandomAffine be applied?

    Augmentation adds variety during training. The test pipeline must be deterministic so that evaluation is reproducible and comparable.
  3. 3. In a confusion matrix with true classes as rows, what does a large number off the diagonal in row 4, column 9 mean?

    Rows are true labels and columns are predictions, so cell (4, 9) counts true 4s that were predicted as 9.
  4. 4. Why is softmax not needed when you only want the predicted class from a classifier's logits?

    Softmax is monotonic, so it never changes which entry is largest. It is only needed when you want actual probabilities.

Summary

  • A vision pipeline is dataset, transforms, DataLoader, model, loss, optimizer and an evaluation pass on unseen data.
  • ToTensor gives floats in 0 to 1 with channels first; Normalize standardizes each channel with training-set statistics (MNIST: 0.1307 and 0.3081).
  • Augment only the training data; keep the test pipeline deterministic.
  • Use model.train() for training and model.eval() plus torch.no_grad() for evaluation; predict with argmax on logits.
  • A confusion matrix, precision and recall show which classes get confused, which accuracy alone hides.

Next, reuse a model somebody else trained in Transfer Learning and Fine-Tuning.