Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 45 ยท Computer Vision

Convolutions and CNNs

Learn how convolutions work and how to build a small CNN in PyTorch: Conv2d, kernels, stride, padding, pooling, output shapes and receptive fields.

A fully connected layer treats an image as a long flat list of numbers. That throws away what makes images special: nearby pixels belong together, and the same pattern (an edge, a corner, a texture) can show up anywhere. A convolutional layer exploits both facts. It slides one small grid of weights, a kernel, across the image and reuses the same weights at every position. The result is far fewer parameters and a model that finds a pattern no matter where it appears.

In this lesson you will learn what nn.Conv2d computes, how stride, padding and pooling change the shape of the data, how to predict output sizes with a formula, and how to stack layers into a small CNN. PyTorch itself cannot run on this site, so the PyTorch code is read-only. The live editors use NumPy: a hand-written convolution (the real arithmetic that Conv2d performs) and a shape calculator. They are stand-ins for the library, not the library, but the numbers are the same.

What a conv layer does

Slide a kernel A small weight grid visits every position.
Multiply and add Each stop gives one output number.
Share weights The same kernel is reused everywhere.
Feature map The outputs form a new image.

Many kernels give many feature maps, one per output channel.

1 / 2

Tensors for images

PyTorch image batches have the layout (N, C, H, W): batch size, channels, height, width. A batch of 32 RGB images of 28 by 28 pixels is a tensor of shape (32, 3, 28, 28). A grayscale image has one channel. This "channels first" order differs from many image libraries (and from NumPy plus Matplotlib images, which are usually height, width, channels), so you will often call permute to convert. See Indexing, Slicing and Reshaping for permute.

What Conv2d computes

Python
import torch
from torch import nn

conv = nn.Conv2d(in_channels=1, out_channels=4, kernel_size=3)
x = torch.randn(8, 1, 28, 28)       # a batch of 8 grayscale images
y = conv(x)
print(y.shape)                      # torch.Size([8, 4, 26, 26])
print(conv.weight.shape)            # torch.Size([4, 1, 3, 3])
print(conv.bias.shape)              # torch.Size([4])

The weight tensor has shape (out_channels, in_channels, kH, kW). Each of the 4 output channels owns one 3 by 3 kernel per input channel. For one output position, the layer multiplies the kernel with the 3 by 3 patch under it, sums everything (across input channels too), and adds the bias. Moving the patch one pixel at a time over a 28 by 28 image gives 26 by 26 positions, hence the output shape above.

One detail surprises people with a maths background: deep learning libraries do not flip the kernel, so what they call convolution is mathematically cross-correlation. Since the kernel values are learned, it makes no difference to what the network can represent.

The parameter count is small. This layer has 4 * 1 * 3 * 3 = 36 weights plus 4 biases, 40 in total, regardless of the image size. A dense layer from 784 pixels to the 2,704 values of the output would need over two million weights.

Convolution by hand

The first editor performs the sliding multiply-and-add in plain NumPy, using two classic kernels that detect vertical and horizontal edges on a tiny image containing a bright square.

Python Sliding a kernel over an image (stand-in for Conv2d)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the response: positive numbers mark places where dark turns bright going left to right, negative numbers mark bright turning dark, and zeros mean no vertical change. A learned CNN does exactly this, except nobody chooses the kernel by hand: gradient descent finds useful ones. Early layers tend to discover edge and color detectors, later layers combine them into shapes and parts.

Stride, padding and the output size formula

Three settings control the output size of a convolution or pooling layer along one dimension:

  • kernel size k: the width of the sliding window;
  • padding p: how many zeros are added to each side;
  • stride s: how many pixels the window moves each step.

For an input of size n, the output size is

out = floor((n + 2p - k) / s) + 1

Check it against the example above: n = 28, p = 0, k = 3, s = 1 gives 26. With padding 1 the formula yields 28 again, which is why padding=1 with a 3 by 3 kernel is called "same" padding: the height and width are preserved. PyTorch also accepts padding="same" for stride 1 layers, and padding="valid" meaning no padding.

Python
nn.Conv2d(3, 16, kernel_size=3, padding=1)             # keeps H and W
nn.Conv2d(16, 32, kernel_size=3, stride=2, padding=1)  # halves H and W
nn.MaxPool2d(kernel_size=2)                            # halves H and W (stride defaults to kernel_size)

Pooling layers use the same formula but have no weights. MaxPool2d(2) keeps the largest value in each 2 by 2 block, so a 28 by 28 map becomes 14 by 14. AvgPool2d averages instead. nn.AdaptiveAvgPool2d(1) is handy at the end of a network: whatever the spatial size, it averages each channel down to a single number, so the classifier does not depend on the input resolution.

The second editor is a shape calculator. It follows an image through a stack of layers and prints the formula's result at every step, together with the parameter count of each convolution.

Python Shape calculator for a stack of layers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Receptive fields

The receptive field of a value is the region of the original image that can influence it. One 3 by 3 conv has a receptive field of 3 by 3. Stack a second 3 by 3 conv and each output sees a 5 by 5 region; a third gives 7 by 7. Strides and pooling make it grow even faster, because later kernels step over larger and larger jumps of the original image. This is why deep stacks of small kernels are preferred: three 3 by 3 layers cover the same 7 by 7 area as one 7 by 7 layer with fewer weights (27 versus 49 per channel pair) and extra non-linearities in between. Deeper layers can therefore recognise bigger things, such as a whole digit, while early layers only see strokes.

A small CNN

Here is a complete model for 28 by 28 grayscale images and 10 classes, written as a PyTorch module (see Modules, Parameters and Layers):

Python
class SmallCNN(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(1, 16, kernel_size=3, padding=1),   # (16, 28, 28)
            nn.ReLU(),
            nn.MaxPool2d(2),                              # (16, 14, 14)
            nn.Conv2d(16, 32, kernel_size=3, padding=1),  # (32, 14, 14)
            nn.ReLU(),
            nn.MaxPool2d(2),                              # (32, 7, 7)
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),                                 # (32 * 7 * 7,) = 1568
            nn.Linear(32 * 7 * 7, num_classes),
        )

    def forward(self, x):
        return self.classifier(self.features(x))

model = SmallCNN()
print(model(torch.randn(4, 1, 28, 28)).shape)   # torch.Size([4, 10])

The pattern is the standard one: a feature extractor of conv, activation and pooling blocks that shrinks the spatial size while growing the number of channels, followed by a classifier that flattens the result and maps it to logits. The outputs are raw scores (logits), so train with nn.CrossEntropyLoss (see Activation Functions and Loss Functions). Add nn.BatchNorm2d after convolutions and nn.Dropout before the final layer to train deeper models more reliably, as covered in Initialization, Normalization and Dropout.

When a Linear layer after Flatten complains about a size mismatch, the culprit is almost always the formula: the flattened size depends on the input image size. Print the shape after the feature extractor once, or use nn.LazyLinear, which infers its input size on the first batch.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the shape of a batch of 16 RGB images of 64 by 64 pixels in PyTorch's default layout?

    PyTorch convolution layers expect (batch, channels, height, width), so the three color channels come right after the batch dimension.
  2. 2. An input has height 32. It goes through Conv2d(kernel_size=5, stride=1, padding=0). What is the output height?

    floor((32 + 0 - 5) / 1) + 1 = 28. Without padding the output shrinks by kernel_size - 1.
  3. 3. Why does a convolutional layer need far fewer parameters than a dense layer on the same image?

    Weight sharing means the parameter count depends on kernel size and channel counts, not on the image size.
  4. 4. Which layer has no learnable parameters?

    Max pooling only picks the largest value in each window. Convolution, linear and batch norm layers all have weights.

Summary

  • Images are tensors of shape (N, C, H, W); Conv2d weights have shape (out_channels, in_channels, kH, kW).
  • A convolution slides a kernel over the input, multiplying and summing at every position, with the same weights everywhere.
  • Output size per dimension: floor((n + 2p - k) / s) + 1; padding=1 with kernel_size=3 keeps the size.
  • Pooling shrinks feature maps without any weights; adaptive pooling makes the head independent of image size.
  • Stacked small kernels grow the receptive field cheaply.
  • A typical CNN is a conv/ReLU/pool feature extractor followed by flatten and a linear classifier that outputs logits.

Next, see how landmark designs stack convolutions in CNN Architectures, Pooling and ResNet, then put a CNN to work in the Image Classification Pipeline.