Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 73 ยท Convolutional Networks

The Convolution Operation

Learn how convolution works: sliding a kernel over an image, padding, stride, the output size formula and edge-detecting filters, all in NumPy.

A fully connected layer looks at every input value at once and gives each one its own weight. For a picture, that is wasteful. A 28 by 28 image already has 784 pixels, and a layer of 100 neurons would need 78,400 weights, yet the interesting things in a picture (an edge, a corner, a curve) are tiny and local, and they look the same wherever they appear. Convolution exploits both facts: it uses a small grid of weights, called a kernel or filter, and slides it across the image, reusing the same weights at every position.

In this lesson you will build the operation from nothing but loops and slicing, learn how padding and stride change the output size, and use hand-made kernels to detect edges. It builds on Images as Tensors, where you saw how an image becomes an array. If you want a purely practical take on filtering with NumPy, Project: Image Filters is a good companion.

Convolution in four moves

Place Lay a small kernel over one patch of the image.
Multiply Multiply each pixel by the weight above it.
Add Sum the products into one output number.
Slide Move the kernel and repeat to fill the output.

The same nine weights are reused at every position.

1 / 2

Sliding a kernel over an image

Take a 5 by 5 image and a 3 by 3 kernel. Place the kernel over the top-left 3 by 3 patch, multiply the nine pairs of numbers, add them up, and write the result into the top-left cell of the output. Then move one column to the right and repeat. When the row ends, drop down one row.

Each output value is a dot product between the kernel and one patch of the image. That is the whole operation. Here it is as a function, applied to a blur kernel (nine weights of 1/9, so each output is the average of its patch):

Python A convolution from scratch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output is 3 by 3, smaller than the 5 by 5 input, and its top-left value (6) is exactly the average of the top-left patch. The output is often called a feature map, because each cell says how strongly its patch matched the kernel.

The output size formula

The kernel can only be placed where it fits entirely inside the image. Along one dimension with input size n, kernel size k, padding p on each side and stride s, the number of valid positions is:

TEXT
output = floor((n + 2p - k) / s) + 1
  • Without padding (p = 0) the output shrinks by k - 1. Stack ten 3 by 3 convolutions and you lose 20 pixels of width, which is a problem for small images.
  • Padding with zeros around the border keeps the size. With an odd kernel, p = (k - 1) / 2 gives a "same" convolution where the output matches the input. Using no padding is called "valid".
  • Stride greater than 1 skips positions, so the output shrinks roughly by a factor of s. Strided convolutions are a common way to downsample.

The formula is easy to get off by one, so check it against the function:

Python Checking the output size formula

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Every row agrees. Notice the "same" rows: kernel 3 with padding 1 and kernel 5 with padding 2 both keep the size at 28. Kernel 3 with padding 1 and stride 2 gives 14, a clean halving.

Kernels that detect edges

What do the weights do? Different weights make different feature detectors. A kernel whose left column is negative and whose right column is positive responds strongly where dark meets bright from left to right, and gives zero on flat regions, where the positives and negatives cancel.

The classic pair is the Sobel kernels:

TEXT
vertical edges        horizontal edges
[-1  0  1]            [-1 -2 -1]
[-2  0  2]            [ 0  0  0]
[-1  0  1]            [ 1  2  1]

Apply both to a synthetic picture: a bright square on a dark background. The picture is generated in code, so the example is fully deterministic.

Python Detecting edges with Sobel kernels

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

On the flat inside of the square the response is 0. At the left edge it is +4 (dark to bright), and at the right edge it is -4 (bright to dark). The sign tells you the direction of the change. The gx panel lights up the left and right sides of the rectangle, the gy panel the top and bottom, and the magnitude panel shows the full outline.

Here lies the central idea of convolutional networks: you do not have to design these kernels by hand. In a CNN, the kernel values are parameters that gradient descent learns, and the network discovers edge detectors (and much more) on its own. A 3 by 3 kernel has only 9 weights, no matter how large the image is.

Convolution or cross-correlation?

There is a small piece of terminology that confuses many learners. In mathematics, true convolution flips the kernel (up-down and left-right) before sliding it. What we computed above, and what every deep learning library calls "convolution", does not flip. Strictly speaking it is cross-correlation.

Since the kernel is learned, flipping makes no difference: if the flipped version is useful, training simply finds the flipped weights. So the deep learning world dropped the flip, and the name stuck. You can see the relationship with SciPy, which offers both:

Python Convolution versus cross-correlation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For a symmetric kernel such as a blur, the two agree. For an asymmetric one they differ, but true convolution equals cross-correlation with a flipped kernel. In this course, as in PyTorch and TensorFlow, "convolution" means the unflipped version.

Doing it without loops

Python loops over every output pixel are slow. NumPy can build a view that contains every patch at once, so the whole operation becomes one multiplication and one sum. The function sliding_window_view returns an array of shape (out_h, out_w, kh, kw) without copying any data (see views and copies).

Python A vectorized convolution

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both versions produce identical numbers, and the vectorized one is many times faster. The next lessons build on this idea: Convolution, Pooling and Feature Maps stacks many kernels and channels, and the training lesson turns the patch extraction into the im2col trick.

Kernels are almost always odd-sized (3, 5, 7) so that they have a center pixel, which makes "same" padding symmetric. Stacking two 3 by 3 convolutions sees the same 5 by 5 area as one 5 by 5 kernel but uses fewer weights (18 instead of 25), which is why 3 by 3 became the default.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A 32 by 32 image goes through a 5 by 5 kernel with padding 0 and stride 1. What is the output size?

    output = (32 + 0 - 5) / 1 + 1 = 28. Without padding, the output shrinks by kernel size minus one.
  2. 2. Which padding gives a 'same' convolution with a 3 by 3 kernel and stride 1?

    With p = (k - 1) / 2 = 1, the formula gives n + 2 - 3 + 1 = n, so the output keeps the input size.
  3. 3. What does a Sobel-style kernel output on a perfectly flat region of an image?

    Edge kernels have weights that sum to zero, so when all pixels in the patch are equal, the products cancel out.
  4. 4. How does deep-learning 'convolution' differ from mathematical convolution?

    Libraries slide the kernel without flipping it. Because the kernel is learned, the flip would make no difference to what the network can represent.

Summary

  • A convolution slides a small kernel over an image and takes a dot product at every position, reusing the same few weights everywhere.
  • The result is a feature map; large values mean the patch resembles the kernel.
  • Output size per dimension is floor((n + 2p - k) / s) + 1. Padding keeps the size, stride downsamples.
  • Hand-made kernels such as Sobel detect edges; in a CNN the kernel values are learned by gradient descent.
  • Deep learning "convolution" is cross-correlation: the kernel is not flipped.
  • sliding_window_view plus einsum replaces the double loop with one fast array operation.

Next: Convolution, Pooling and Feature Maps.